Note: As usual, readers of my blog (and my students) found a simpler and more elegant solution than I did… I knew that the boolean “or” operator returned the first “True” value it encountered, but I completely forgot that if it only encounters “False” values, it returns the final one. So there was a far more elegant solution than I suggested, namely “[LIST COMPREHENSION] or None”. I’ll leave this blog post up, since it was still fun to explore these ideas… but as usual, when you get too complex in Python, you’re probably overlooking a simpler and more straightforward solution, as I did here.
A few weeks ago, I held my monthly “office hours” session for subscribers to Weekly Python Exercise. WPE students are always invited not only to ask questions about what we’re learning in the course, but also any other Python-related issue that they have encountered.
Well, someone asked quite a doozy this month: He said that he wants to use a list comprehension to create a list for a project at work. Except that if the list comprehension is empty, then he wants to get a “None” value.
In other words: He wants a list comprehension, or at the very least an expression containing one, which will either return a list (if non-empty) or “None” (if the list is empty).
In answering this question, I managed to pull together what might be the greatest collection of unreadable Python constructs in a single expression. I’m not recommending that you write this sort of code — but it does demonstrate that Python’s syntax does lend itself to all sorts of creative solutions and possibilities, if you know how to combine things.
Let’s start by pointing out that a list comprehension always returns a list. Regardless of how many elements it might contain, the result of a list comprehension is always going to be a list. For example:
>>> [x*x for x in range(5)] [0, 1, 4, 9, 16] >>> [x*x for x in range(0)] 
In both of the above cases, a list value was returned; there’s no such thing as a list comprehension that returns a non-list value. Even an empty list is a list, after all.
My student would thus need to accept that while a list comprehension could be part of the solution, it couldn’t be the entire solution. We would need something like an if-else statement. For example:
mylist = [x*x for x in range(5)] if mylist: output = mylist else: output = None
The above code will certainly work, and would be my preferred way to solve such a problem. But for whatever reason, my student said that we needed to use a single expression; an if-else statement wouldn’t suffice.
Fortunately, Python does offer an inline, expression version of if-else. I personally find it hard to read and understand, but it’s designed for situations like this one, in which we need a conditional expression. It looks like this:
TRUE_OUTPUT if CONDITION else FALSE_OUTPUT
In other words, it’s a one-line “if-else” expression, returning one value if the condition is met, and another value if it is not. For example:
>>> 'Yes' if True else 'No' 'Yes' >>> 'Yes' if False else 'No' 'No'
Of course, we can have any expression that we might like. So we could say:
>>> mylist = [x*x for x in range(5)] >>> mylist if len(mylist) > 0 else None [0, 1, 4, 9, 16]
In other words: If “mylist” is non-empty, then we’ll get “mylist” back. Otherwise, we’ll get “None” back. And it works!
However, it’s considered un-Pythonic to check for an empty list (or any other empty data structure) by checking its length. Rather, we can check to see if it’s empty simply by putting “mylist” in an “if” statement. In a boolean context, all lists (as well as strings, tuples, and dicts) return “True” so long as they contain any values, but “False” if they’re empty. We can thus rewrite the above code as:
>>> mylist = [x*x for x in range(5)] >>> mylist if mylist else None [0, 1, 4, 9, 16]
This is fine, but it’s not a single expression, which was a requirement. Fortunately, we can just squish everything into a single line, replacing any reference to “mylist” with the list comprehension itself:
>>> [x*x for x in range(5)] if [x*x for x in range(5)] else None [0, 1, 4, 9, 16]
Now, this is getting pretty ugly. Among other things, we have repeated our list comprehension twice in the same line. After all, our one-line “if-else” expression is just that, an expression, with no assignment allowed. So if we want to keep things on a single line, only using expressions, there’s no way for us to store the output from our list comprehension for later, is there?
There wasn’t. But then came Python 3.8 with the “assignment expression” operator, aka “the walrus,” which changed everything. The walrus is designed to be used in just this kind of situation. OK, maybe not quite this ugly of an expression, but it can help us to get out of such pickles.
What I can do is use the walrus operator to capture the list created by the list comprehension. We can then use the variable to which we’ve assigned our list, thus saving us from having to use the list comprehension a second time.
Note that the one-line “if-else” is confusing on several fronts, but nowhere more so than the fact that the condition (in the middle of the expression) executes first, before either of the output expressions is evaluated. This makes sense, when you think about it. but it can still be confusing to put an assignment in the middle of a line, so that it can be used at the start of the line.
So, let’s try it:
output if output := [x*x for x in range(5)] else None
This doesn’t work, and that’s because we need to put the central expression inside of parentheses to ensure that Python’s parser knows what is going on:
>>> output if (output := [x*x for x in range(5)]) else None [0, 1, 4, 9, 16]
It worked! Thanks to a combination of the condition expression, the walrus operator, list comprehensions, and the fact that empty lists are “False” in a boolean context, we managed to get a single expression that returns the result of a list comprehension when it contains values, and “None” otherwise.
And while I would question the wisdom of having such code in an actual production system, I freely admit that there are times when such hacks are necessary. And despite the fact that Python has a more rigid syntax than many other languages, its functional parts made it possible for us to achieve our goal with only a minimum of code.
It’s time: My Black Friday sale, with 40% off all 25 products in my online store, is live!
What am I offering?
How do you take advantage of this? Just use coupon code BF2020 at checkout. Or click on this link, which will apply it automatically at checkout.
This is just a reminder that on Sunday, I’ll be teaching a live, 4-hour introductory course on databases and SQL. If you haven’t ever worked with databases before, then this will give you the push that you need to understand how they work, and how to work effectively with them.
I’ve been using SQL for 30 years, but it’s far from an outdated skill. No only do many of the world’s largest companies run on databases using SQL, but the smartphone in your pocket does, too. Moreover, once you understand the basics, you can start to ask serious and interesting questions from public and private datasets.
In this class, I’ll introduce you to SQL via the open-source PostgreSQL database. There will be numerous exercises and many opportunities to ask questions. And after the course is over, you’ll have access to an exclusive forum in which you can ask me questions about SQL.
Sounds good? Sign up here: https://store.lerner.co.il/intro-to-sql
I’ve gotten a lot of e-mail with questions from people curious about the course. So without further ado, here’s a FAQ:
None; SQL is a query language, not a programming language. It’ll help to be able to think logically, which programming certainly helps to develop. But we won’t be using any language other than SQL itself.
You’ll understand how relational databases work in a general way. But beyond that, you’ll be able to:
Probably not. We’ll be doing basic queries — the sort of thing that everyone needs to know when working with databases, but nothing too fancy.
That said, I’ve gotten enough questions like this one that I’ll almost certainly do an “Advanced SQL” course in the coming months.
No, this course is only about SQL. We won’t be using Python! However, just as I’ll be doing an advanced SQL course in the coming months, I’ll probably offer one on using databases from within Python. But you’ll need the knowledge in this course to take that one.
I’ll be setting up a virtual machine with PostgreSQL installed, along with the pre-loaded data sets we’ll be using. And I’ll give you a copy of that when we’re done, so that you can practice at home. We’ll likely use a Web-based interface; I’ll give you full details before the course opens. And if you need more help, then I’ll provide it in class and in our forum.
When I did my “pytest” course a few months ago, several people suggested that I provide either office hours or a forum after the course had ended, in case people had follow-up questions. This course will thus have a forum, open to anyone who has registered to ask questions after the course is over. I thus don’t see it as a one-and-done class, but a chance to ask questions and keep learning after it’s over.
Just as my Python courses go beyond the syntax and help you to really understand what’s going on behind the scenes, this course will help you to think in terms of databases and tables. You’ll understand not just the “what” but the “why.” And when the course is over, you’ll have the foundation you need to learn and do more on your own.
But wait, there’s more: Because this is a live course, you’ll have a chance to ask questions and get detailed explanations. My goal is to really help you learn, and if that means explaining something several times until you get it, then that’s what I’ll do. This isn’t a generic course; it’s one that will cater to your specific needs.
Absolutely! My standard discount policy applies:
Otherwise, learn more and sign up at https://store.lerner.co.il/intro-to-sql. I look forward to seeing you on Sunday!
Have you heard? Data is “the new oil” — meaning, data is the most valuable and important thing in the modern world. Which means that if you can store, retrieve, and organize your data, then you (and your company) are positioned for greater success.
This usually means working with a database — and frequently, a relational database, with which you communicate using a language called SQL.
In other words: SQL is the key to the modern data revolution. But too often, people are put off from learning SQL. It seems weird, even when compared with a programming language.
Well, I have good news: If you want to join the data revolution and work with databases, I’m offering a new course. On November 15th, I’ll be teaching a live, 4-hour online course, “Intro to SQL.” I’ll teach you the basics of what you need to work with a database.
The course includes:
I’ve been using databases since 1995, and have been teaching SQL for more than 20 years. This course is based on that corporate training, and is meant to get you jump started into the world of data and relational databases. We’ll be using PostgreSQL, a powerful open-source database I’ve been using for more than two decades.
Questions? Learn more at https://store.lerner.co.il/intro-to-sql (where there’s an extensive FAQ). Or contact me on Twitter (@reuvenmlerner) or via e-mail (firstname.lastname@example.org). I’ll answer as soon as I can.
I hope to see you there!
Over the last year, I’ve gotten increasingly active on my YouTube channel, https://YouTube.com/reuvenlerner. Each week, I upload 1-2 new videos, typically answering questions that I’ve gotten in my corporate training classes or from people online — via e-mail, or on Twitter (@reuvenmlerner).
So if you’re looking to learn about Jupyter shortcuts, or inner classes in Python, or the differences between “modules” and “packages,” then head on over to https://YouTube.com/reuvenlerner, and subscribe! And if there are Python topics you would like me to address, don’t hesitate to contact me. You might see your question answered in a video!
Want to improve your Python skills? Looking for a way to practice on a regular basis, backed up by a community of learners?
Look no more: A new advanced-level cohort of Weekly Python Exercise is starting tomorrow! If you’ve been using Python for at least a year, then this course will open your eyes to new techniques, and help to strengthen existing ones.
Here’s how it works:
But wait, there’s more: As of this cohort (B3), every solution will not only be written up in e-mail, but will also be answered in a screencast! I hope that this will help you to understand the solutions better than in pure text.
But don’t hesitate; I won’t be offering this cohort again until 2021…
If you’ve been using Python for a year or so, then you’re no longer confused or surprised by the language’s basics — the core data structures, functions, and even basic object-oriented programming. But you probably don’t quite feel fluent with Python, and aren’t sure how to use some of the language’s more advanced features. It would sure be nice to understand these things better, not just by reading a blog, but via actual, hands-on practice.
If that describes you, then you should check out Weekly Python Exercise, the course that helps you to level up your Python skills. A new advanced-level cohort (B3) starts on Tuesday, October 27th, and works as follows:
(If you’re a Python beginner, then a new A-level cohort be starting in January. You can learn more at https://WeeklyPythonExercise.com/.)
Want to know more, or see some sample exercises? Go to https://store.lerner.co.il/wpe-b3.
Later this month, I’ll appear on the “Exploiting with Teja Kummarikuntla” podcast. As part of that appearance, I’ll be doing an AMA (“ask me anything”) segment — but in order for that to happen, I need questions!
That’s where you come in: If there’s a question that you would like for me to answer, then please go ahead and submit it at https://exploit.chat/AskLerner. Possible topics include:
If you have questions on other topics, then go ahead and submit those, too!
I’m really looking forward to appearing on the podcast, and to answering your questions.
I’ve been a professional programmer for about 30 years, self-employed for 25 years, and doing full-time corporate Python training for more than a decade.
I run a small business, which involves me writing, programming, and teaching, as well as handling all of the business-related stuff.
So, what’s my most important skill, the thing that helps me get lots accomplished in a short period of time? Easy: My ability to touch type.
It all started when I was in high school in the mid-1980s. I would use my family’s computer — yes, in those days, the entire family shared one — for schoolwork, for doing some introductory programming, and even writing newsletters for my high-school youth organization. The thing is, I was doing all of this typing with two fingers, and this drove my parents bananas.
Both of my parents can touch type. In those days, it was typical for office workers to record their correspondence, give the recording to a secretary, and then review the result before sending it out. My father never did that, because he typed at least as fast as his secretary, and the whole dictation process would slow him down. It wasn’t unusual to hear the rat-tat-tat of my father typing from his study at home.
It’s no surprise that it bothered my parents to be hunting and pecking. I was pretty fast at it, but I was no match for my father or any other touch typist. My parents strongly encouraged me to learn to touch type, but I was a teenager, which meant that I knew better than they did. And besides, I type fast enough, right?
Finally, my parents set a new rule: For every hour that I used the computer, I had to spend an hour doing a lesson from a touch-typing book. (How quaint, right?) I yelled. I screamed. I cried. I protested. But my parents didn’t budge.
At first, it was painful: When you start to touch type, you are learning to use your hands in a new way, one that feels completely foreign. You also type much more slowly than you did before, and feel like you’re wasting your time. I certainly had these feelings, and when I had to get something done quickly, I would refer to my old two-finger method.
But within two or three weeks, I was already touch typing as quickly as I did with two fingers. Better yet, and somewhat amazingly, I was able to type without looking at the keyboard! I could enter passages from a book, without having to move my eyes from book to keyboard and back. I could talk to someone while typing. I could even sneak a peak at the TV while I was typing.
Achieving true speed didn’t happen for a while. But when I started college in the fall of 1988, I was already typing at a pretty fast clip. At the student newspaper, I was frequently drafted to take printouts from the Associated Press and type them into our “world and nation” section. And at the computer labs, where we had loud, mechanical IBM keyboards, people would ask me if I could type more slowly, because the rat-tat-tat was disturbing them.
Fast forward to 2020, and I cannot imagine my work without being able to touch type:
Lots of professional writers know that they need to touch type. After all, they write for a living, and being unable to get the most out of their keyboard would seem like a crazy thing to do.
And yet, I find that a small number of the developers in my courses can touch type. They never really thought about it that much, or decided not to put time and effort into it, or thought that it was hard or impossible to learn. But it’s definitely not a priority.
Touch typing looks magical and impossible to achieve. It’s like watching a virtuoso pianist expressing themselves through the instrument, their thoughts and feelings flowing effortlessly from their brains to their hands, and then to the piano.
But here’s the thing: It’s not hard to learn. You’ll be frustrated for the weeks during which you’re learning and forcing yourself to work in a new way. But it pays for itself in spades, allowing you to write, edit, and express yourself — in code and text — more easily than you could ever imagine. And if I managed to learn from a book as an angry teenager, then you can certainly learn with the variety of online tools, many of them free, available today.
Whether you’re a developer, devops engineer, or data scientist, you’re likely using Python. But do you really know the language, or do you find yourself copying and pasting from Stack Overflow on a regular basis, hoping that the solution you’ve found will solve your problem without too much editing?
The best solution to this problem is practice. And in my book, Python Workout (https://PythonWorkout.com/), I provide you with 50 exercises (and another 150 bonus exercises) to push your Python skills forward, helping you not just to solve the problem at hand but to generally understand how the language works.
Python Workout, along with two other books (“Tiny Python Projects” and “Data Science Bookcamp”) is currently 50% off, as a Manning “Deal of the day.” No matter what you’re doing with Python, you’ll likely benefit from or more of these books. Just go to https://manning.com/dotd and enter the code dotd100620 at checkout to get half off.
But don’t wait — this deal is only good on October 6, 2020!