You can, but should you? Combining some of Python’s more esoteric features

Note: As usual, readers of my blog (and my students) found a simpler and more elegant solution than I did… I knew that the boolean “or” operator returned the first “True” value it encountered, but I completely forgot that if it only encounters “False” values, it returns the final one. So there was a far more elegant solution than I suggested, namely “[LIST COMPREHENSION] or None”. I’ll leave this blog post up, since it was still fun to explore these ideas… but as usual, when you get too complex in Python, you’re probably overlooking a simpler and more straightforward solution, as I did here.

A few weeks ago, I held my monthly “office hours” session for subscribers to Weekly Python Exercise. WPE students are always invited not only to ask questions about what we’re learning in the course, but also any other Python-related issue that they have encountered.

Well, someone asked quite a doozy this month: He said that he wants to use a list comprehension to create a list for a project at work. Except that if the list comprehension is empty, then he wants to get a “None” value.

In other words: He wants a list comprehension, or at the very least an expression containing one, which will either return a list (if non-empty) or “None” (if the list is empty).

In answering this question, I managed to pull together what might be the greatest collection of unreadable Python constructs in a single expression. I’m not recommending that you write this sort of code — but it does demonstrate that Python’s syntax does lend itself to all sorts of creative solutions and possibilities, if you know how to combine things.

Let’s start by pointing out that a list comprehension always returns a list. Regardless of how many elements it might contain, the result of a list comprehension is always going to be a list. For example:

>>> [x*x for x in range(5)]
[0, 1, 4, 9, 16]

>>> [x*x for x in range(0)]
[]

In both of the above cases, a list value was returned; there’s no such thing as a list comprehension that returns a non-list value. Even an empty list is a list, after all.

My student would thus need to accept that while a list comprehension could be part of the solution, it couldn’t be the entire solution. We would need something like an if-else statement. For example:

mylist = [x*x for x in range(5)]

if mylist:
    output = mylist
else:
    output = None

The above code will certainly work, and would be my preferred way to solve such a problem. But for whatever reason, my student said that we needed to use a single expression; an if-else statement wouldn’t suffice.

Fortunately, Python does offer an inline, expression version of if-else. I personally find it hard to read and understand, but it’s designed for situations like this one, in which we need a conditional expression. It looks like this:

TRUE_OUTPUT if CONDITION else FALSE_OUTPUT

In other words, it’s a one-line “if-else” expression, returning one value if the condition is met, and another value if it is not. For example:

>>> 'Yes' if True else 'No'
'Yes'
>>> 'Yes' if False else 'No'
'No'

Of course, we can have any expression that we might like. So we could say:

>>> mylist = [x*x for x in range(5)]
>>> mylist if len(mylist) > 0 else None
[0, 1, 4, 9, 16]

In other words: If “mylist” is non-empty, then we’ll get “mylist” back. Otherwise, we’ll get “None” back. And it works!

However, it’s considered un-Pythonic to check for an empty list (or any other empty data structure) by checking its length. Rather, we can check to see if it’s empty simply by putting “mylist” in an “if” statement. In a boolean context, all lists (as well as strings, tuples, and dicts) return “True” so long as they contain any values, but “False” if they’re empty. We can thus rewrite the above code as:

>>> mylist = [x*x for x in range(5)]
>>> mylist if mylist else None
[0, 1, 4, 9, 16]

This is fine, but it’s not a single expression, which was a requirement. Fortunately, we can just squish everything into a single line, replacing any reference to “mylist” with the list comprehension itself:

>>> [x*x for x in range(5)] if [x*x for x in range(5)] else None
[0, 1, 4, 9, 16]

Now, this is getting pretty ugly. Among other things, we have repeated our list comprehension twice in the same line. After all, our one-line “if-else” expression is just that, an expression, with no assignment allowed. So if we want to keep things on a single line, only using expressions, there’s no way for us to store the output from our list comprehension for later, is there?

There wasn’t. But then came Python 3.8 with the “assignment expression” operator, aka “the walrus,” which changed everything. The walrus is designed to be used in just this kind of situation. OK, maybe not quite this ugly of an expression, but it can help us to get out of such pickles.

What I can do is use the walrus operator to capture the list created by the list comprehension. We can then use the variable to which we’ve assigned our list, thus saving us from having to use the list comprehension a second time.

Note that the one-line “if-else” is confusing on several fronts, but nowhere more so than the fact that the condition (in the middle of the expression) executes first, before either of the output expressions is evaluated. This makes sense, when you think about it. but it can still be confusing to put an assignment in the middle of a line, so that it can be used at the start of the line.

So, let’s try it:

output if output := [x*x for x in range(5)] else None

This doesn’t work, and that’s because we need to put the central expression inside of parentheses to ensure that Python’s parser knows what is going on:

>>> output if (output := [x*x for x in range(5)]) else None
[0, 1, 4, 9, 16]

It worked! Thanks to a combination of the condition expression, the walrus operator, list comprehensions, and the fact that empty lists are “False” in a boolean context, we managed to get a single expression that returns the result of a list comprehension when it contains values, and “None” otherwise.

And while I would question the wisdom of having such code in an actual production system, I freely admit that there are times when such hacks are necessary. And despite the fact that Python has a more rigid syntax than many other languages, its functional parts made it possible for us to achieve our goal with only a minimum of code.