Функциональное программирование в Python: lambda, zip, filter, map reduce
Функциональным называется такой подход к процессу программирования, в программа рассматривается как вычисление математических функций, при этом не используются состояния и изменяемые объекты. Как правило, когда говорят о элементах функционального программировании в Python, то подразумеваются следующие функции: lambda, map, filter, reduce, zip.
Lambda выражение в Python:
lambda оператор или lambda функция в Python это способ создать анонимную функцию, то есть функцию без имени. Такие функции можно назвать одноразовыми, они используются только при создании. Как правило, lambda функции используются в комбинации с функциями filter, map, reduce.
Синтаксис lambda выражения в Python
В качестве arguments передается список аргументов, разделенных запятой, после чего над переданными аргументами выполняется expression. Если присвоить lambda-функцию переменной, то получим поведение как в обычной функции (делаем мы это исключительно в целях демонстрации)
Но, конечно же, все преимущества lambda-выражений мы получаем, используя lambda в связке с другими функциями
Функция map() в Python:
В Python функция map принимает два аргумента: функцию и аргумент составного типа данных, например, список. map применяет к каждому элементу списка переданную функцию. Например, вы прочитали из файла список чисел, изначально все эти числа имеют строковый тип данных, чтобы работать с ними — нужно превратить их в целое число:
Тот же эффект мы можем получить, применив функцию map:
Как видите такой способ занимает меньше строк, более читабелен и выполняется быстрее. map также работает и с функциями созданными пользователем:
А теперь то же самое, только используя lambda выражение:
Функция map может быть так же применена для нескольких списков, в таком случае функция-аргумент должна принимать количество аргументов, соответствующее количеству списков:
Если же количество элементов в списках совпадать не будет, то выполнение закончится на минимальном списке:
Функция filter() в Python:
Функция filter предлагает элегантный вариант фильтрации элементов последовательности. Принимает в качестве аргументов функцию и последовательность, которую необходимо отфильтровать:
Обратите внимание, что функция, передаваемая в filter должна возвращать значение True / False, чтобы элементы корректно отфильтровались.
Функция reduce() в Python:
Функция reduce принимает 2 аргумента: функцию и последовательность. reduce() последовательно применяет функцию-аргумент к элементам списка, возвращает единичное значение. Обратите внимание в Python 2.x функция reduce доступна как встроенная, в то время, как в Python 3 она была перемещена в модуль functools.
Вычисление суммы всех элементов списка при помощи reduce:
Вычисление наибольшего элемента в списке при помощи reduce:
Функция zip() в Python:
Функция zip объединяет в кортежи элементы из последовательностей переданных в качестве аргументов.
Обратите внимание, что zip прекращает выполнение, как только достигнут конец самого короткого списка.
Python's reduce(): From Functional to Pythonic Style
Python’s reduce() is a function that implements a mathematical technique called folding or reduction. reduce() is useful when you need to apply a function to an iterable and reduce it to a single cumulative value. Python’s reduce() is popular among developers with a functional programming background, but Python has more to offer.
In this tutorial, you’ll cover how reduce() works and how to use it effectively. You’ll also cover some alternative Python tools that can be more Pythonic, readable, and efficient than reduce() .
In this tutorial, you’ll learn:
- How Python’s reduce() works
- What the more common reduction use cases are
- How to solve these use cases using reduce()
- What alternative Python tools are available to solve these same use cases
With this knowledge, you’ll be able to decide which tools to use when it comes to solving reduction or folding problems in Python.
For a better understanding of Python’s reduce() , it would be helpful to have some previous knowledge of how to work with Python iterables, especially how to loop over them using a for loop.
Free Download: Get a sample chapter from Python Tricks: The Book that shows you Python’s best practices with simple examples you can apply instantly to write more beautiful + Pythonic code.
Exploring Functional Programming in Python
Functional programming is a programming paradigm based on breaking down a problem into a set of individual functions. Ideally, every function only takes a set of input arguments and produces an output.
In functional programming, functions don’t have any internal state that affects the output that they produce for a given input. This means that anytime you call a function with the same set of input arguments, you’ll get the same result or output.
In a functional program, input data flows through a set of functions. Each function operates on its input and produces some output. Functional programming tries to avoid mutable data types and state changes as much as possible. It works with the data that flow between functions.
Other core features of functional programming include the following:
- The use of recursion rather than loops or other structures as a primary flow control structure
- A focus on lists or arrays processing
- A focus on what is to be computed rather than on how to compute it
- The use of pure functions that avoid side effects
- The use of higher-order functions
There are several important concepts in this list. Here’s a closer look to some of them:
Recursion is a technique in which functions call themselves, either directly or indirectly, in order to loop. It allows a program to loop over data structures that have unknown or unpredictable lengths.
Pure functions are functions that have no side effects at all. In other words, they’re functions that do not update or modify any global variable, object, or data structure in the program. These functions produce an output that depends only on the input, which is closer to the concept of a mathematical function.
Higher-order functions are functions that operate on other functions by taking functions as arguments, returning functions, or both, as with Python decorators.
Since Python is a multi-paradigm programming language, it provides some tools that support a functional programming style:
- Functions as first-class objects capabilities
- Anonymous functions with lambda and generators
- Standard modules like functools and itertools
- Tools like map() , filter() , reduce() , sum() , len() , any() , all() , min() , max() , and so on
Even though Python isn’t heavily influenced by functional programming languages, back in 1993 there was a clear demand for some of the functional programming features listed above.
In response, several functional tools were added to the language. According to Guido van Rossum, they were contributed by a community member:
Python acquired lambda , reduce() , filter() and map() , courtesy of (I believe) a Lisp hacker who missed them and submitted working patches. (Source)
Over the years, new features such as list comprehensions, generator expressions, and built-in functions like sum() , min() , max() , all() , and any() were viewed as Pythonic replacements for map() , filter() , and reduce() . Guido planned to remove map() , filter() , reduce() , and even lambda from the language in Python 3.
Luckily, this removal didn’t take effect, mainly because the Python community didn’t want to let go of such popular features. They’re still around and still widely used among developers with a strong functional programming background.
In this tutorial, you’ll cover how to use Python’s reduce() to process iterables and reduce them to a single cumulative value without using a for loop. You’ll also learn about some Python tools that you can use in place of reduce() to make your code more Pythonic, readable, and efficient.
Getting Started With Python’s reduce()
Python’s reduce() implements a mathematical technique commonly known as folding or reduction. You’re doing a fold or reduction when you reduce a list of items to a single cumulative value. Python’s reduce() operates on any iterable—not just lists—and performs the following steps:
- Apply a function (or callable) to the first two items in an iterable and generate a partial result.
- Use that partial result, together with the third item in the iterable, to generate another partial result.
- Repeat the process until the iterable is exhausted and then return a single cumulative value.
The idea behind Python’s reduce() is to take an existing function, apply it cumulatively to all the items in an iterable, and generate a single final value. In general, Python’s reduce() is handy for processing iterables without writing explicit for loops. Since reduce() is written in C, its internal loop can be faster than an explicit Python for loop.
Python’s reduce() was originally a built-in function (and still is in Python 2.x), but it was moved to functools.reduce() in Python 3.0. This decision was based on some possible performance and readability issues.
Another reason for moving reduce() to functools was the introduction of built-in functions like sum() , any() , all() , max() , min() , and len() , which provide more efficient, readable, and Pythonic ways of tackling common use cases for reduce() . You’ll learn how to use them in place of reduce() later in the tutorial.
In Python 3.x, if you need to use reduce() , then you first have to import the function into your current scope using an import statement in one of the following ways:
- import functools and then use fully-qualified names like functools.reduce() .
- from functools import reduce and then call reduce() directly.
According to the documentation for reduce() , the function has the following signature:
The Python documentation also states that reduce() is roughly equivalent to the following Python function:
Like this Python function, reduce() works by applying a two-argument function to the items of iterable in a loop from left to right, ultimately reducing iterable to a single cumulative value .
Python’s reduce() also accepts a third and optional argument called initializer that provides a seed value to the computation or reduction.
In the next two sections, you’ll take an in-depth look at how Python’s reduce() works and the meaning behind each of its arguments.
The Required Arguments: function and iterable
The first argument to Python’s reduce() is a two-argument function conveniently called function . This function will be applied to the items in an iterable to cumulatively compute a final value.
Even though the official documentation refers to the first argument of reduce() as “a function of two arguments,” you can pass any Python callable to reduce() as long as the callable accepts two arguments. Callable objects include classes, instances that implement a special method called __call__() , instance methods, class methods, static methods, and functions.
Note: For more details about Python callable objects, you can check out the Python documentation and scroll down to “Callable types.”
The second required argument, iterable , will accept any Python iterable, as its name suggests. This includes lists, tuples, range objects, generators, iterators, sets, dictionary keys and values, and any other Python objects that you can iterate over.
Note: If you pass an iterator to Python’s reduce() , then the function will need to exhaust the iterator before you can get a final value. So, the iterator at hand won’t remain lazy.
To understand how reduce() works, you’re going to write a function that computes the sum of two numbers and prints the equivalent math operation to the screen. Here’s the code:
This function calculates the sum of a and b , prints a message with the operation using an f-string, and returns the result of the computation. Here’s how it works:
my_add() is a two-argument function, so you can pass it to Python’s reduce() along with an iterable to compute the cumulated sum of the items in the iterable. Check out the following code that uses a list of numbers:
When you call reduce() , passing my_add() and numbers as arguments, you get an output that shows all the operations that reduce() performs to come up with a final result of 10 . In this case, the operations are equivalent to ((((0 + 1) + 2) + 3) + 4) = 10 .
The call to reduce() in the above example applies my_add() to the first two items in numbers ( 0 and 1 ) and gets 1 as the result. Then reduce() calls my_add() using 1 and the next item in numbers (which is 2 ) as arguments, getting 3 as the result. The process is repeated until numbers runs out of items and reduce() returns a final result of 10 .
The Optional Argument: initializer
The third argument to Python’s reduce() , called initializer , is optional. If you supply a value to initializer , then reduce() will feed it to the first call of function as its first argument.
This means that the first call to function will use the value of initializer and the first item of iterable to perform its first partial computation. After this, reduce() continues working with the subsequent items of iterable .
Here’s an example in which you use my_add() with initializer set to 100 :
Since you supply a value of 100 to initializer , Python’s reduce() uses that value in the first call as the first argument to my_add() . Note that in the first iteration, my_add() uses 100 and 0 , which is the first item of numbers , to perform the calculation 100 + 0 = 100 .
Another point to note is that, if you supply a value to initializer , then reduce() will perform one more iteration than it would without an initializer .
If you’re planning to use reduce() to process iterables that may potentially be empty, then it’s good practice to provide a value to initializer . Python’s reduce() will use this value as its default return value when iterable is empty. If you don’t provide an initializer value, then reduce() will raise a TypeError . Take a look at the following example:
If you call reduce() with an empty iterable , then the function will return the value supplied to initializer . If you don’t supply an initializer , then reduce() will raise a TypeError when processing empty iterables.
Note: To dive deeper into what the Python traceback is, check out Understanding the Python Traceback.
Now that you’re familiar with how reduce() works, you’re ready to learn how to apply it to some common programming problems.
Reducing Iterables With Python’s reduce()
So far, you’ve learned how Python’s reduce() works and how to use it to reduce iterables using a user-defined function. You also learned the meaning of each argument to reduce() and how they work.
In this section, you’ll look at some common use cases for reduce() and how to solve them using the function. You’ll also learn about some alternative Python tools that you can use in place of reduce() to make your code more Pythonic, efficient, and readable.
Summing Numeric Values
The «Hello, World!» of Python’s reduce() is the sum use case. It involves calculating the cumulative sum of a list of numbers. Say you have a list of numbers like [1, 2, 3, 4] . Its sum will be 1 + 2 + 3 + 4 = 10 . Here’s a quick example of how to solve this problem using a Python for loop:
The for loop iterates over every value in numbers and accumulates them in total . The final result is the sum of all the values, which in this example is 10 . A variable used like total in this example is sometimes called an accumulator.
This is arguably the most common use case for Python’s reduce() . To implement this operation with reduce() , you have several options. Some of them include using reduce() with one of the following functions:
- A user-defined function
- A lambda function
- A function called operator.add()
To use a user-defined function, you need to code a function that adds two numbers. Then you can use that function with reduce() . For this example, you can rewrite my_add() as follows:
my_add() adds two numbers, a and b , and returns the result. With my_add() in place, you can use reduce() to calculate the sum of the values in a Python iterable. Here’s how:
The call to reduce() applies my_add() to the items in numbers to compute their cumulative sum. The final result is 10 , as expected.
You can also perform the same computation by using a lambda function. In this case, you need a lambda function that takes two numbers as arguments and returns their sum. Take a look at the following example:
The lambda function takes two arguments and returns their sum. reduce() applies the lambda function in a loop to compute the cumulative sum of the items in numbers .
Likewise, you can take advantage of a Python module called operator . This module exports a bunch of functions that correspond to Python’s intrinsic operators. For the problem at hand, you can use operator.add() along with Python’s reduce() . Check out the following example:
In this example, add() takes two arguments and returns their sum. So, you can use add() with reduce() to compute the sum of all the items of numbers . Since add() is written in C and optimized for efficiency, it may be your best choice when using reduce() for solving the sum use case. Note that the use of operator.add() is also more readable than using a lambda function.
The sum use case is so common in programming that Python, since version 2.3, has included a dedicated built-in function, sum() , to solve it . sum() is declared as sum(iterable[, start]) .
start is an optional argument to sum() and defaults to 0 . The function adds the value of start to the items of iterable from left to right and returns the total. Take a look at the following example:
Since sum() is a built-in function, you don’t need to import anything. It’s always available for you. Using sum() is the most Pythonic way of solving the sum use case. It’s clean, readable, and concise. It follows a core Python principle:
The addition of sum() to the language was a big win in terms of readability and performance as compared to using reduce() or a for loop.
Note: For more details on comparing the performance of reduce() with the performance of other Python reduction tools, check out the section Performance is Key.
If you’re dealing with the sum use case, then good practice recommends the use of sum() .
Multiplying Numeric Values
The product use case of Python’s reduce() is quite similar to the sum use case, but this time the operation is multiplication. In other words, you need to calculate the product of all the values in an iterable.
For example, say you have the list [1, 2, 3, 4] . Its product will be 1 * 2 * 3 * 4 = 24 . You can calculate this using a Python for loop. Check out the following example:
The loop iterates over the items in numbers , multiplying each item by the result of the previous iteration. In this case, the starting value for the accumulator product should be 1 instead of 0 . Since any number multiplied by zero is zero, a starting value of 0 will always make your product equal to 0 .
This computation is also a quite popular use case for Python’s reduce() . Again, you’ll cover three ways for solving the problem. You’ll use reduce() with:
- A user-defined function
- A lambda function
- A function called operator.mul()
For option 1, you’ll need to code a custom function that takes two arguments and returns their product. Then you’ll use this function with reduce() to calculate the product of the items in an iterable. Take a look at the following code:
The function my_prod() multiplies two numbers, a and b . The call to reduce() iterates over the items of numbers and computes their product by applying my_prod() to successive items. The final result is the product of all the items in numbers , which in this example is 24 .
If you prefer to use a lambda function to solve this use case, then you need a function that takes two arguments and returns their product. Here’s an example:
The anonymous function does the magic by multiplying successive items while reduce() iterates over numbers . Again, the result is the product of all the items in numbers .
You can also use operator.mul() to tackle the product use case. operator.mul() takes two numbers and returns the result of multiplying them. This is the right functionality for solving the problem at hand. Check out the following example:
Since mul() is highly optimized, your code will perform better if you use this function rather than a user-defined function or a lambda function. Note that this solution is much more readable as well.
Finally, if you’re using Python 3.8, then you have access to a more Pythonic and readable solution to this use case. Python 3.8 has added a new function called prod() , which lives in the Python math module. This function is analogous to sum() but returns the product of a start value multiplied by an iterable of numbers.
In the case of math.prod() , the argument start is optional and defaults to 1 . Here’s how it works:
This is also a big win in terms of readability and efficiency as compared to using reduce() . So, if you’re using Python 3.8 and product reduction is a common operation in your code, then you’ll be better served by using math.prod() rather than Python’s reduce() .
Finding the Minimum and Maximum Value
The problem of finding the minimum and maximum value in an iterable is also a reduction problem that you can solve using Python’s reduce() . The idea is to compare the items in the iterable to find the minimum or the maximum value.
Say you have the list of numbers [3, 5, 2, 4, 7, 1] . In this list, the minimum value is 1 and the maximum value is 7 . To find these values, you can use a Python for loop. Check out the following code:
Both loops iterate over the items in rest and update the value of min_value or max_value according to the result of successive comparisons. Note that initially, min_value and max_value hold the number 3 , which is the first value in numbers . The variable rest holds the remaining values in numbers . In other words, rest = [5, 2, 4, 7, 1] .
Note: In the above examples, you use the Python iterable unpacking operator ( * ) to unpack or expand the values in numbers into two variables. In the first case, the net effect is that min_value gets the first value in numbers , which is 3 , and rest collects the remaining values in a list.
Check out the details in the following examples:
The Python iterable unpacking operator ( * ) is useful when you need to unpack a sequence or iterable into several variables.
For a better understanding of unpacking operations in Python, you can check out PEP 3132 Extended Iterable Unpacking and PEP 448 Additional Unpacking Generalizations.
Now, think about how you can find the minimum and maximum value in an iterable using Python’s reduce() . Again, you can use a user-defined function or a lambda function depending on your needs.
The following code implements a solution that uses two different user-defined functions. The first function will take two arguments, a and b , and return their minimum. The second function will use a similar process, but it’ll return the maximum value.
Here are the functions and how you can use them with Python’s reduce() to find the minimum and maximum value in an iterable:
When you run reduce() with my_min_func() and my_max_func() , you get the minimum and maximum value in numbers , respectively. reduce() iterates over the items of numbers , compares them in cumulative pairs, and finally returns the minimum or maximum value.
Note: To implement my_min_func() and my_max_func() , you used a Python conditional expression, or ternary operator, as a return value. For a deeper dive into what conditional expression are and how they work, check out Conditional Statements in Python (if/elif/else).
You can also use a lambda function to solve the minimum and maximum problem. Take a look at the following examples:
This time, you use two lambda functions that find out if a is either less than or greater than b . In this case, Python’s reduce() applies the lambda function to each value in numbers , comparing it with the result of the previous computation. At the end of the process, you get the minimum or maximum value.
The minimum and maximum problem is so common in programming that Python has added built-in functions to perform these reductions. These functions are conveniently called min() and max() , and you don’t need to import anything to be able to use them. Here’s how they work:
When you use min() and max() to find the minimum and maximum item in an iterable, your code is way more readable as compared to using Python’s reduce() . Additionally, since min() and max() are highly-optimized C functions, you can also say that your code will be more efficient.
So, when it comes to solving this problem in Python, it’s best to use min() and max() rather than reduce() .
Checking if All Values Are True
The all-true use case of Python’s reduce() involves finding out whether or not all the items in an iterable are true. To solve this problem, you can use reduce() along with a user-defined function or a lambda function.
You’ll start by coding a for loop to find out if all the items in an iterable are true. Here’s the code:
If all of the values in iterable are true, then check_all_true() returns True . Otherwise, it returns False . It also returns True with empty iterables. check_all_true() implements a short-circuit evaluation. This means that the function returns as soon as it finds a false value without processing the rest of the items in iterable .
To solve this problem using Python’s reduce() , you’ll need to write a function that takes two arguments and returns True if both arguments are true. If one or both arguments are false, then the function will return False . Here’s the code:
This function takes two arguments, a and b . Then you use the and operator to test if both arguments are true. The return value will be True if both arguments are true. Otherwise, it’ll be False .
In Python, the following objects are considered false:
like None and False
- Numeric types with a zero value like 0 , 0.0 , 0j , Decimal(0) , and Fraction(0, 1)
- Empty sequences and collections like «» , () ,  , <> , set() , and range(0)
- Objects that implement __bool__() with a return value of False or __len__() with a return value of 0
Any other object will be considered true.
You need to use bool() to convert the return value of and into either True or False . If you don’t use bool() , then your function won’t behave as expected because and returns one of the objects in the expression instead of True or False . Check out the following examples:
and returns the first value in the expression if it’s false. Otherwise, it returns the last value in the expression regardless of its truth value. That’s why you need to use bool() in this case. bool() returns the Boolean value ( True or False ) resulting from evaluating a Boolean expression or an object. Check out the examples using bool() :
bool() will always return either True or False after evaluating the expression or object at hand.
Note: To better understand Python operators and expressions, you can check out Operators and Expressions in Python.
You can pass both_true() to reduce() to check if all the items of an iterable are true or not. Here’s how this works:
If you pass both_true() as an argument to reduce() , then you’ll get True if all of the items in the iterable are true. Otherwise, you’ll get False .
In the third example, you pass True to the initializer of reduce() to get the same behavior as check_all_true() and to avoid a TypeError .
You can also use a lambda function to solve the all-true use case of reduce() . Here are some examples:
This lambda function is quite similar to both_true() and uses the same expression as a return value. It returns True if both arguments are true. Otherwise, it returns False .
Note that unlike check_all_true() , when you use reduce() to solve the all-true use case, there’s no short-circuit evaluation because reduce() doesn’t return until it traverses the entire iterable. This can add extra processing time to your code.
For example, say you have the list lst = [1, 0, 2, 0, 0, 1] and you need to check if all the items in lst are true. In this case, check_all_true() will finish as soon as its loop processes the first pair of items ( 1 and 0 ) because 0 is false. You don’t need to continue iterating because you already have an answer for the problem at hand.
On the other hand, the reduce() solution won’t finish until it processes all the items in lst . That’s five iterations later. Now imagine what this would do to the performance of your code if you were processing a large iterable!
Fortunately, Python provides the right tool for solving the all-true problem in a Pythonic, readable, and efficient way: the built-in function all() .
You can use all(iterable) to check if all of the items in iterable are true. Here’s how all() works:
all() loops over the items in an iterable, checking the truth value of each of them. If all() finds a false item, then it returns False . Otherwise, it returns True . If you call all() with an empty iterable, then you get True because there’s no false item in an empty iterable.
all() is a C function that’s optimized for performance. This function is also implemented using short-circuit evaluation. So, if you’re dealing with the all-true problem in Python, then you should consider using all() instead of reduce() .
Checking if Any Value Is True
Another common use case for Python’s reduce() is the any-true use case. This time, you need to find out if at least one item in an iterable is true. To solve this problem, you need to write a function that takes an iterable and returns True if any item in the iterable is true and False otherwise. Take a look at the following implementation for this function:
If at least one item in iterable is true, then check_any_true() returns True . It returns False only if all the items are false or if the iterable is empty. This function also implements a short-circuit evaluation because it returns as soon as it finds a true value, if any.
To solve this problem using Python’s reduce() , you need to code a function that takes two arguments and returns True if at least one of them is true. If both are false, then the function should return False .
Here’s a possible implementation for this function:
any_true() returns True if at least one of its arguments it true. If both arguments are false, then any_true() returns False . As with both_true() in the above section, any_true() uses bool() to convert the result of the expression a or b to either True or False .
The Python or operator works a little differently from and . It returns the first true object or the last object in the expression. Check out the following examples:
The Python or operator returns the first true object or, if both are false, the last object. So, you also need to use bool() to get a coherent return value from any_true() .
Once you have this function in place, you can continue with the reduction. Take a look at the following calls to reduce() :
You’ve solved the problem using Python’s reduce() . Note that in the third example, you pass False to the initializer of reduce() to reproduce behavior of the original check_any_true() and also to avoid a TypeError .
Note: Like the examples in the previous section, these examples of reduce() don’t make a short-circuit evaluation. That means they can affect the performance of your code.
You can also use a lambda function with reduce() to solve the any-true use case. Here’s how you can do it:
This lambda function is quite similar to any_true() . It returns True if either of its two arguments is true. If both arguments are false, then it returns False .
Even though this solution takes only one line of code, it can still make your code unreadable or at least difficult to understand. Again, Python provides a tool to efficiently solve the any-true problem without using reduce() : the built-in function any() .
any(iterable) loops over the items in iterable , testing the truth value of each until it finds a true item. The function returns True as soon as it finds a true value. If any() doesn’t find a true value, then it returns False . Here’s an example:
Again, you don’t need to import any() to use it in your code. any() works as expected. It returns False if all the items in the iterable are false. Otherwise, it returns True . Note that if you call any() with an empty iterable, then you get False because there’s no true item in an empty iterable.
As with all() , any() is a C function optimized for performance. It’s also implemented using short-circuit evaluation. So, if you’re dealing with the any-true problem in Python, then consider using any() instead of reduce() .
Comparing reduce() and accumulate()
A Python function called accumulate() lives in itertools and behaves similarly to reduce() . accumulate(iterable[, func]) accepts one required argument, iterable , which can be any Python iterable. The optional second argument, func , needs to be a function (or a callable object) that takes two arguments and returns a single value.
accumulate() returns an iterator. Each item in this iterator will be the accumulated result of the computation that func performs. The default computation is the sum. If you don’t supply a function to accumulate() , then each item in the resulting iterator will be the accumulated sum of the previous items in iterable plus the item at hand.
Check out the following examples:
Note that the last value in the resulting iterator is the same value that reduce() returns. This is the main similarity between these two functions.
Note: Since accumulate() returns an iterator, you need to call list() to consume the iterator and get a list object as an output.
If, on the other hand, you supply a two-argument function (or callable) to the func argument of accumulate() , then the items in the resulting iterator will be the accumulated result of the computation performed by func . Here’s an example that uses operator.mul() :
In this example, you can again see that the last item in the returned value of accumulate() is equal to the value returned by reduce() .
Considering Performance and Readability
Python’s reduce() can have remarkably bad performance because it works by calling functions multiple times. This can make your code slow and inefficient. Using reduce() can also compromise the readability of your code when you use it with complex user-defined functions or lambda functions.
Throughout this tutorial, you’ve learned that Python offers a bunch of tools that can gracefully replace reduce() , at least for its main use cases. Here are the main takeaways of your reading up to this point:
Use a dedicated function to solve use cases for Python’s reduce() whenever possible. Functions such as sum() , all() , any() , max() , min() , len() , math.prod() , and so on will make your code faster and more readable, maintainable, and Pythonic.
Avoid complex user-defined functions when using reduce() . These kinds of functions can make your code difficult to read and understand. You can use an explicit and readable for loop instead.
Avoid complex lambda functions when using reduce() . They can also make your code unreadable and confusing.
The second and third points were concerns for Guido himself when he said the following:
So now reduce() . This is actually the one I’ve always hated most, because, apart from a few examples involving + or * , almost every time I see a reduce() call with a non-trivial function argument, I need to grab pen and paper to diagram what’s actually being fed into that function before I understand what the reduce() is supposed to do. So in my mind, the applicability of reduce() is pretty much limited to associative operators, and in all other cases it’s better to write out the accumulation loop explicitly. (Source)
The next two sections will help you implement this general advice in your code. They also provide some extra advice that will help you use Python’s reduce() effectively when you really need to use it.
Performance Is Key
If you’re going to use reduce() to solve the use cases that you’ve covered in this tutorial, then your code will be considerably slower as compared to code using dedicated built-in functions. In the following examples, you’ll use timeit.timeit() to quickly measure the execution time of small bits of Python code and get an idea of their general performance.
timeit() takes several arguments, but for these examples, you’ll only need to use the following:
- stmt holds the statement that you need to time.
- setup takes additional statements for general setup, like import statements.
- globals holds a dictionary containing the global namespace that you need to use for running stmt .
Take a look at the following examples that time the sum use case using reduce() with different tools and using Python’s sum() for comparison purposes:
Even though you’ll get different numbers depending on your hardware, you’ll likely get the best time measurement using sum() . This built-in function is also the most readable and Pythonic solution for the sum problem.
Note: For more a detailed approach to how to time your code, check out Python Timer Functions: Three Ways to Monitor Your Code.
Your second-best option would be to use reduce() with operator.add() . The functions in operator are written in C and are highly optimized for performance. So, they should perform better than a user-defined function, a lambda function, or a for loop.
Code readability is also an important concern when it comes to using Python’s reduce() . Even though reduce() will generally perform better than a Python for loop, as Guido himself stated, a clean and Pythonic loop is often easier to follow than using reduce() .
The What’s New In Python 3.0 guide reinforces this idea when it says the following:
Use functools.reduce() if you really need it; however, 99 percent of the time an explicit for loop is more readable. (Source)
To better understand the importance of readability, imagine that you’re starting to learn Python and you’re trying to solve an exercise about calculating the sum of all the even numbers in an iterable.
If you already know about Python’s reduce() and have done some functional programming in the past, then you might come up with the following solution:
In this function, you use reduce() to cumulatively sum the even numbers in an iterable. The lambda function takes two arguments, x and y , and returns their sum if they’re even. Otherwise, it returns x , which holds the result of the previous sum.
Additionally, you set initializer to 0 because otherwise your sum will have an initial value of 1 (the first value in iterable ), which isn’t an even number and will introduce a bug into your function.
The function works as you expected, and you’re happy with the result. However, you continue digging into Python and learn about sum() and generator expressions. You decide to rework your function using these new tools, and your function now looks as follows:
When you look at this code, you feel really proud, and you should. You’ve done a great job! That’s a beautiful Python function that almost reads as plain English. It’s also efficient and Pythonic. What do you think?
Python’s reduce() allows you to perform reduction operations on iterables using Python callables and lambda functions. reduce() applies a function to the items in an iterable and reduces them to a single cumulative value.
In this tutorial, you’ve learned:
- What reduction, or folding, is and when it might be useful
- How to use Python’s reduce() to solve common reduction problems like summing or multiplying numbers
- Which Pythonic tools you can use to effectively replace reduce() in your code
With this knowledge, you’ll be able to decide which tools best fit your coding needs when it comes to solving reduction problems in Python.
Over the years, reduce() has been replaced by more Pythonic tools like sum() , min() , max() all() , any() , among others. However, reduce() is still there and is still popular among functional programmers. If you have questions or thoughts about using reduce() or any of its Python alternatives, then be sure to post them in the comments below.
Get a short & sweet Python Trick delivered to your inbox every couple of days. No spam ever. Unsubscribe any time. Curated by the Real Python team.
About Leodanis Pozo Ramos
Leodanis is an industrial engineer who loves Python and software development. He's a self-taught Python developer with 6+ years of experience. He's an avid technical writer with a growing number of articles published on Real Python and other sites.
Each tutorial at Real Python is created by a team of developers so that it meets our high quality standards. The team members who worked on this tutorial are:
How to use the reduce() method in Python
The reduce() function facilitates a functional approach to Python programming. It performs a rolling-computation as specified by the passed function to the neighboring elements, by taking a function and an iterable as arguments, and returns the final computed value.
Note: A rolling-computation is the one where we compute the required value by going through all of the data starting from the first value, where each new result depends on the last computed result of the previous data.
The reduce() function computation is similar to a for-loop in Python, but being an in-built function, it’s much better and faster.
Python Reduce Function Usage
Reduce() seems to be a Python method that performs the folding or compression of the mathematical approach. When you’re about to implement a method over an iterable and limit it to a singular cumulative value, reduce() comes in handy. Python’s reduce() method is famous among functional programmers, although Python has many more to offer. You’ll learn how to use reduce() functions and how to implement them efficiently in this article.
Installation of Python
Before we indulge in the learning of reducing methods in python via the Ubuntu Linux system, we need to have python installed and configured on our Linux system. So to start the learning, we should log in first from the login screen of the Ubuntu system. After the login, launch your command-line shell by activity area or using the shortcut “Ctrl+Alt+T” key. You have to check whether your system has python installed already or not. And, for this purpose, try out the query below. You can see it is now showing the latest version. For that, we have to check its version that has already been installed through the below command.
Now, you need to update the apt package that has already been mounted on your Linux system to update python again. For updation of the apt package, use the stated-below query in the shell.
After the updation of the apt package, you need to download the additional packages via the ppa package throughout the below query needed for additional python updates.
Tap on the Enter key to continue the installation.
After downloading additional packages and updates, we will mount the new version of the Python repository in the Ubuntu System. Hence, we have been trying the beneath query in the shell as follows:
Tap “Y” to carry on the procedure of setting up, or else knock out the key “n” to end it. Now the updated version of Python has been sufficiently configured on your system. Now, another package should be installed and configured on your system to use python on it. This package is named “pip.” This is necessary to be installed before the usage of python. As the pip package is not installed, we will be installing it via curl command. Therefore, install the curl package first as:
After the installation of curl, download the package of pip via:
You can have a look at the list of all the repositories beginning with “get.”
Using the sudo command with the keyword “python3.9”, we will extract the “py” file below. Please wait for a while until it gets completed.
The last line of the output shows that the latest pip package is installed and mounted on Ubuntu 20.04 System.
You can have a look at the latest installed version of the package “pip” as follows:
Reduce Example 01:
Let’s have a first example of reduce function to see its working. First of all, open the terminal and create a new .py file named “one.py” using the touch query as follows:
Now go to the File Explorer and direct towards the home folder. Open the newly created one.py file and write the script shown below in it. This script contains the function of reducing in it. Firstly, you need to import the reduce method from the library functools to use this function in our code using the “from” keyword. After that, we have declared a method “sum” to calculate a sum of two variables, “a” and “b,” and return this sum to a calling statement. After that, we have initialized a list of integers, and a print statement has been used to apply the reduce function. We have passed the list to a function “sum,” and the “sum” has been passed to the reduce method. You can see we have applied the sum function to the list “l.” So this will sum up the values of the list and return them to the print statement to be shown. Save the file.
Let’s execute the saved file by using the “python3” keyword command while using the name of a file “one.py” along with it and tap on the “Enter” button. The terminal’s output shows the sum of all the integer values of a list as 24.
Reduce Example 02:
Let’s take another simple yet lengthy example of reduce method in python to understand it more. Open the same file “one.py” from the home directory and update the code with the below script. Now, this script contains library functools that have been importing the reduce method first. After that, we have defined a function factorial with 2 parameters, x and y, to calculate the factorial of integers provided in some time. The print statement is printing the integer variables and return the calculated factorial or multiplication of both the variables to the calling statement. Then we have defined a 5 element list “l.” The next two print statements have been used to print string values and the sum of the list elements, consequently using the lambda function within the reduce function to avoid complications. The next two print statements are defined to check the largest value from the list using lambda using the reducing method. The last two print statements have been used to print and calculate the factorial by using list members.
We have found the sum, the largest value, and the factorial of the list values in a sequence without any complexity by executing the file.
Hopefully! You will find this article helpful and easy to implement on your side. Now you can easily use python reduce function by implementing different examples of your own choice.
About the author
I am a self-motivated information technology professional with a passion for writing. I am a technical writer and love to write for all Linux flavors and Windows.