Let’s de-confuse Python objects!

Are you confused about object-oriented Python? You’re not alone. In teaching Python to companies around the world for more than 20 years, I’ve found that almost everyone is confused by Python objects:

  • Newcomers to object-oriented programming are just plain ol’ confused — by the terminology, the syntax, and why we even need to use objects.
  • People with experience using objects in other language wonder why Python’s objects are so weird. Why do we need to use “self” so much? And what’s with the static fields that behave strangely?
  • Even people with Python experience, who have been using objects for a while, find the object system to be a confusing mish-mash of seemingly consistent rules.

Sounds like you? Well, I have good news: Starting this Sunday, I’ll help you make sense of it all in live, online classes about object-oriented Python.

When you’re done, you’ll not only have a clearer mental model. You’ll also be able to write shorter, more expressive, more idiomatic, more maintainable code.

In other words, you’ll be able to do more in less time, and with less code. That’ll help you in your current job — and can improve your chances of getting a new, better job.

So, what am I teaching?

September 12th: Intro Python Objects

  • Classes
  • Instances
  • Attributes
  • Class attributes
  • Inheritance
  • Basic magic methods

September 19th: Advanced Python Objects

  • Advanced magic methods
  • Context managers
  • Multiple inheritance
  • Metaclasses
  • Properties
  • Descriptors

I’m running these classes like I do my corporate training: With lots of time for Q&A, and a ton of hands-on exercises. Without slides, as I live-code into Jupyter. We’ll even have a private forum, so you can ask questions when the class is done. And of course, you’ll have access to the video recording.

If you’re confused about Python objects, or want to take your understanding to the next level, then these classes are for you.

Questions? Comments? Eligible for a discount (for students, retirees/pensioners, or people living outside the world’s 30 richest countries)? Hit me up on Twitter (@reuvenmlerner) or e-mail (reuven@lerner.co.il), and I’ll answer ASAP.

Reuven

PS: These are two of the six live courses I’m running in September and October. Take a look at all six topics here: https://store.lerner.co.il/liveclasses

Missing my blog posts? Check me out on YouTube!

I haven’t been posting much to this blog of late, in part because I’ve been posting Python-related videos on YouTube. Just in the last month or two, I’ve posted:

I plan to write more on this blog as well, but don’t forget to check out my YouTube channel for these Python lessons. And all of the Jupyter notebooks I use in that channel are available on GitHub for you to download, at https://github.com/reuven/youTube-notebooks.

If you have topics you want me to explore, just e-mail me at reuven@lerner.co.il, or DM me on Twitter at @reuvenmlerner.

Level up your Python skills with six new courses

Want to level up your Python skills, to solve more problems in less time, and with less code?

Want to take advantage of the latest techniques in the Python world?

Want to ensure that your code is as idiomatic as possible, so that whoever has to maintain it down the line (including you) will have an easier time working with it?

Well, good news: In September and October, I’ll be teaching six different live Python courses that will help you to achieve those goals:

  • September 12th: Intro Python objects — a live version of one of my most popular recorded and corporate-training courses, introducing object-oriented programming in Python. We’ll talk about (among other things) classes, instances, attributes, and inheritance.
  • September 19th: Advanced Python objects — again, a live version of a popular recorded class, showing how objects work behind the scenes and some advanced techniques for working with them, including such topics as magic methods, properties, and descriptors.
  • September 26th: Functional programming in Python — Learn about various functional techniques in Python, including partial functions, dispatch tables, and such functions as map, filter, and reduce.
  • October 3rd: Threading and multiprocessing —  Want to know how to use threads and processes to add concurrency to your programs? We’ll discuss both of these techniques, including their pros and cons.
  • October 10th: asyncio — The newest form of Python concurrency, ascynio, is different from threading and multiprocessing. Learn what it is, how to use it, and how to write clients and servers using asyncio.
  • October 17th: Creating a Python project — Your goal in writing Python code isn’t just to write the code, but rather to create something that you can distribute to others, either as a library or application. In this class, we’ll discuss the structure of a project, and how to distribute it.

Want to buy access to all six courses?  You can do this with this course bundle — which also gives you a 15% discount, or the ability to pay in installments: https://store.lerner.co.il/bundle-six-live-courses

Several additional points:

  • This is the same sort of class that I’ve taught on-site for Fortune 500 companies for 20 years (and online to many of them since the pandemic started), full of examples, clear explanations, and bad jokes.
  • You’ll have a chance to ask me questions while they’re running.
  • I’m going to set up a forum for each class, so that if you have questions once the class is done, you’ll still be able to ask me, and/or be in touch with other participants.
  • Every class will include numerous hands-on exercises.
  • Each class will start at 6 p.m. in Israel, which is 11 a.m. Eastern, and will go for about 4 hours.
  • Recordings will be available within about 24 hours of the class taking place.  You’ll have access to that recording forever.
  • You’ll also have access to any files, Jupyter notebooks, and programs I produce while teaching the class.
  • As always, I offer discounts to students, seniors/pensioners, people permanently living outside of the world’s 30 richest countries, and also anyone who has been affected by the covid/coronavirus pandemic. Just reply to this message, and I’ll give you details.

Questions or comments? Just contact me at reuven@lerner.co.il or as @reuvenmlerner on Twitter. I’ll be happy to answer.

To learn more about each class, and to sign up, check out the “live classes” section of my online store: https://store.lerner.co.il/live-classes.

I look forward to seeing you in one or more of these classes!

Improve your “pandas” skills with “Pandas Workout”

As you might know, I’ve been working lately on my new book, Pandas Workout, with 50 exercises to improve your fluency using the “pandas” library for data analysis.

Well, I have exciting news: The book is now available as a MEAP, aka the “Manning Early Access Program.” This means that if you buy the book now, you’ll have access to the chapters as they’re written.

The text needs further editing, as well as technical reviews, graphics, diagrams, indexing, and the like. So it’s not done. But it’s definitely in good enough shape to start improving your “pandas” chops, allowing you to understand how the library works and how to use it better.

Also: The “50 exercises” description is a bit incorrect: Every exercise has three “Beyond the exercise” problems, taking the existing question to the next level or asking you to perform a related task. So when I say “50 exercises,” I really mean, “200 exercises.” But hey, what’s a factor of 4 among friends?

Even better: I’ve created Jupyter notebooks for all of the exercise solutions, including the “beyond” exercises — so you can download and experiment with those on your own computer.

But wait, there’s more!

To celebrate the release of this MEAP, Manning has provided a 50% discount. Just use the coupon code “mllerner250” at checkout.

You can get a copy of Pandas Workout at https://PandasWorkout.com/.

And hey, while you’re at it — you can also check out Python Workout, at https://PythonWorkout.com/.

Questions? Just e-mail me at reuven@lerner.co.il, or hit me up on Twitter as @reuvemmlerner.

And now? It’s time to get back to writing…

1

Python parameters primer

Python makes it easy to write functions. For example, I can write:

def hello(name):
    return f'Hello, {name}!'

I can then run the function with:

hello('world')

which will then, not surprisingly, return the string

'Hello, world'

Here’s a question that doesn’t come up much: How does Python assign the string argument ‘world’ to the parameter “name”? That is, how does Python know that the value “world” should be assigned to the variable “name”?

You might be thinking, “It’s obvious. I mean, there’s only one value, and there’s only one variable. How else could it work?”

But of course, programming languages don’t work based on what we think is obvious. They have rules, and follow those rules to the letter. So, what are the rules for assigning arguments to parameters?

It turns out that this question isn’t as trivial as it seems. And that’s because while function parameters in Python don’t indicate what types of values they can accept, they do tell Python what types of arguments they can accept — that is, the ways in which they can (or cannot) be assigned arguments when the function is called. (And yes, type hints sorta kinda let us indicate what types of values can be assigned to parameters, but the Python language itself ignores these.)

In this blog post, I want to review the seven (!) different types of parameters, and how Python assigns arguments to them.

Positional vs. keyword arguments

Before we can talk about parameters, we have to talk about arguments. And yes, most developers will confuse these two terms, or use them interchangeably. Simply put, an argument is a value that is passed to a function, whereas a parameter is a variable to which we assign a value. Parameters are local variables, and work just like all other local variables in Python.

When you call a function, you can pass values as two different types of arguments:

  • Positional arguments, where the position of the argument in the call indicates which parameter should get its value
  • Keyword arguments, which always have the form name=value, tell the receiving function which parameter (name) should be assigned the argument (value).

That’s it: There are no other types of arguments in Python! Everything is either positional or keyword. Parameters can indicate how they’re willing to be assigned arguments.

Simple parameters

The simplest, and most recognizable, parameters in Python are the ones without any special decorations. For example, the “hello” function from the top of this blog post has a single, simple parameter:

def hello(name):
return f'Hello, {name}!'

I can call this function with a single positional argument:

hello('world')

Or I can call this function with a single keyword argument:

hello(name='world')

In the first case, Python sees that we have passed a single positional argument. It checks the function’s definition to see how many arguments we will accept (__code__.co_argcount in the function object) and then grabs the parameter’s name (__code__.co_varnames in the code object). Python thus knows that argument #1 should be assigned to parameter #1 — meaning, “world” should be assigned to “name”.

In the second case, Python sees that we have passed a single keyword argument, with a name “name” and a value “world”. It checks (again, with __code__.co_argcount and __code__.co_varnames) whether there is a parameter called “name”. Once it discovers that there is, it assigns “world” to the variable “name”, and all is good.

What happens if we pass the wrong number of positional arguments? Python notices this, and raises an exception, telling us how many arguments the function expects to get, and how many we actually passed. (We’ll get back to this in a little bit.) For example:

>>> hello('a', 'b', 'c')
Traceback (most recent call last):
File "", line 1, in 
TypeError: hello() takes 1 positional argument but 3 were given         

What happens if we pass keyword arguments for parameters that don’t exist? Python notices this as well, and raises an exception. For example:

>>> hello(a=10, b=20, c=30)
Traceback (most recent call last):
File "", line 1, in 
TypeError: hello() got an unexpected keyword argument 'a'

Multiple parameters

What if a function has multiple parameters? Then we can invoke it with positional arguments, keyword arguments, or even a mixture of these. For example, let’s take the following function:

def calc(first, op, second):
   if op == '+':
       return first + second
   elif op == '-':
       return first - second
   else:
       return f'Operator {op} is unsupported'

I can call this function in a variety of ways:

  • Only with positional arguments, as in “calc(10, ‘+’, 3)”
  • Only with keyword arguments, as in “calc(first=10, op=’+’, second=3)”
  • Keyword arguments in a different order, as in “calc(second=3, first=10, op=’+’)”.
  • A mixture of positional and keyword arguments, as in “calc(10, op=’+’, second=3)”

This last example is perhaps the most interesting of the bunch. That’s because Python always allows you to mix positional and keyword arguments, so long as all of the positional arguments come before all of the keyword arguments. Here’s what happens if you get this wrong:

>>> calc(first=10, op='+', 3)
File "", line 1
calc(first=10, op='+', 3)
                       ^
SyntaxError: positional argument follows keyword argument

If a function has these sorts of simple parameters, then we can call the function with any combination of positional and keyword arguments, so long as the number of arguments matches the number of parameters and all of the positional arguments come before the keyword arguments.

Parameters with defaults

Sometimes, we want to give a parameter a default value, so that the caller doesn’t need to provide an argument. We can do this with a syntax that’s identical to that of keyword arguments, name=value, except that this is done in the function definition, not in the function call. For example, we can rewrite our “calc” function as follows:

def calc(first, op='+', second=0):
   if op == '+':
       return first + second
   elif op == '-':
       return first - second
   else:
       print(f'Operator {op} is unsupported')

The above function is identical to the previous version, except that we’ve now indicated that the caller is only required to pass one argument, which will be assigned to the parameter “first”. Both “op” and “second” are now optional, because if we don’t pass them any arguments, they’ll get the default values “+” and 0.

This is possible because when we create our function, Python creates an attribute, “__defaults__”, on the newly created function object. __defaults__ is a tuple containing all of the default values. When we call the function, it checks __code__.co_argcount to find out how many arguments we’re supposed to pass — and if we’re missing one or more, Python grabs the missing one(s) from __defaults__.

In order to make this work, parameters with defaults must be defined after those that lack defaults. That makes it possible for Python to figure out which parameters didn’t get values from the caller, and thus need to get values from __defaults__.

For example, __defaults__ in the “calc” function is (‘+’, 0). So if I invoke “calc(5)”, Python sees that we need to pass three arguments. But we’ve only passed one, so it needs to retrieve the final two values from calc.__defaults__ and then assign those values to “op” and “second”. After it does that, it invokes the function — whose body is blissfully unaware of how this happened.

It’s important to point out that the default values are set when the function is defined, and thus compiled. For this reason, you can get into big, big trouble if you use a mutable default, such as a list. For example:

def add_one(x=[]):
    x.append(1)
    return x

You might think that the default value in the above function means: If I call “add_one” without any arguments, then x will be assigned to an empty list. But no! The list is created when we define the function, and stored away in __defaults__. This means that every time we call the function without any arguments, Python will grab the list from __defaults__ and assign it to x. The function will modify the list, which not only affects x, but also affects the list as stored in the __defaults__ tuple. In other words, our empty list will not be very empty after several calls to the function.

The moral of this story is: Never use mutable default values in a function. Code checkers like “pylint” will notice this for you, but once you realize just how bad this can be, you will likely avoid it without any help.

So far, both of these parameters types can be assigned arguments via either positional or keyword arguments. Now let’s shake things up a bit, though, and talk about the weirder and more unusual parameter types.

*args (aka: all unclaimed positional arguments)

Many people are familiar with the term “*args”, and even use it a bit — but I’ve found that they often don’t understand just what it’s for, or how to use it.

If you have a parameter whose name is preceded by “*”, then its type will be a tuple, and it will contain all of the positional arguments that weren’t assigned to another parameter. Traditionally, this parameter is called “*args”, and pronounced “splat-args,” on the assumption that the “*” sign looks something like a squashed bug. You can call this parameter whatever you want, though — and I often do.

Once your function has *args as a parameter, there is no maximum number of positional arguments that it can take. Any extras, as it were, will be placed into *args. For that reason, *args must be the last parameter that can take positional arguments. (We’ll see some others which can only take keyword arguments in a little bit.) If there are any parameters preceding *args, then *args will be assigned those arguments that weren’t grabbed by the other parameters.

I should note that there isn’t really a technical advantage to use *args. You could, of course, just have a single parameter that takes a list or tuple. But there’s something that feels natural about passing a bunch of values as individual arguments, rather than passing a list or tuple. For example, if I have a function that sums numbers, then it might make more sense to pass he numbers as separate arguments than to pass a list. If I have a function that performs a network security check on IP addresses, then I could take the addresses as positional arguments, rather than as a list. And if I have a function that summarizes data from files, it would make sense to have the function take filenames (strings), rather than a list of filenames.

For example, we can write a number-summing function as follows:

def mysum(numbers):
    total = 0
    for one_number in numbers:
        total += one_number
    return total

To run the above function, I need to pass an iterable of numbers, as in “mysum([10, 20, 30])”. That’s because the function, as defined, takes a single argument (either positional or keyword). But if I change the function definition ever so slightly:

def mysum(*numbers):
    total = 0
    for one_number in numbers:
        total += one_number
    return total

Now the function expects to get only positional arguments, and any number of them. The parameter (i.e., variable) “numbers” will now be a tuple, containing whatever values were passed to the function. I can now call:

  • mysum(10, 20, 30)
  • mysum(1,2,3,4,5)
  • mysum() # no elements, returns 0
  • mysum(*[10, 20 ,30]) # a bit advanced — turns a list into individual arguments!

What if the function has other parameters that accept positional arguments? Those must be before *args. And we can have as many of those as we like. For example:

def myfunc(a, b, *args):
    return f'{a=}, {b=}, {args=}'

If I call “myfunc(10, 20, 30, 40, 50)”, then we’ll see that a is 10, b is 20, and args is the tuple (30, 40, 50).

The typical use for *args is in a “for” loop, list comprehension, or similar iteration. If your function accepts *args but then pulls out index 0, 1, or 2 specifically, then you are probably using it wrong. Similarly, you probably don’t want to be calling “len” on the tuple. The idea is that you’ll do the same thing with all of the arguments, no matter how many (or few) there might be.

What happens if I want my function to have a parameter with a default, and then take *args? For example:

def myfunc(a, b=10, *args):
    return f'{a=}, {b=}, {args=}'

If I run “myfunc(5,6,7,8,9)”, then it’s pretty obvious that a will be 5, b will be 6, and args will be the tuple (7,8,9). But what if I want to pass a value for “a”, pass additional positional arguments to “args”, and have the default value of “b” in place?

Answer: You can’t. There isn’t any way to do that. Because along the way to *args, you have to give “b” a value. (We’ll see in a bit how we can use keyword-only parameter to accomplish this.)

You might be thinking: Of course there’s a way to do this! I can just call the function, passing “args” as a keyword argument. Then I can sidestep “b” and give a value to “args”. But that won’t work, because you would be assigning to “args” as a keyword argument, and args (by definition) only takes positional values. You cannot use “args” as the name in a keyword argument; Python will tell you that it’s not aware of such a parameter.

**kwargs (aka: all unclaimed keyword arguments)

Just as *args allows a function to accept as many positional arguments as we might want to pass, **kwargs allows a function to accept as many keyword arguments as we might like. As with *args, the name “kwargs” is traditional but not mandatory; you can use any name you want. The variable “kwargs” is a dict in which the keys are strings, and the values are whatever was passed in the keyword arguments For example:

def myfunc(a, b=10, **kwargs):
   return f'{a=}, {b=}, {kwargs=}'

If I call “myfunc(5, 6, 7, 8)”, then I’ll get an error — because the function only accepts up to two positional arguments. All the rest must be keyword arguments! So I can try “myfunc(5,6, x=100, y=200)” and then we’ll see the following output:

"a=5, b=6, kwargs={'x': 100, 'y': 200}"

Notice that our keyword arguments (x and y) were turned into strings. Both 100 and 200 remained as integers, as they were passed.

Also notice that we don’t have the problem with “b” that we did before, with *args. We can just call “myfunc(5, x=100, y=200)”, and we’ll get:

"a=5, b=10, kwargs={'x': 100, 'y': 200}"

See? We got the default value for “b”, because there weren’t any positional arguments to override it.

Keyword-only parameters

We saw earlier that we cannot have a parameter with a default before our *args parameter. Fortunately, Python 3 introduced a new kind of parameter: The keyword-only parameter. Any parameter that comes after *args can only be set with keyword arguments.

There are actually two types of keyword-only parameters: If you define it as name=value, then the parameter has a default value, and doesn’t need to be specified by the caller. But if you just include the parameter name, without a default value, then the keyword argument is mandatory, and calling the function without it will result in an error.

For example:

def myfunc(a, *args, b, c=10):
…: return f'{a=}, {b=}, {c=}, {args}'

If I call it with “myfunc(5,6,7,8,9)”, then we get the following error:

myfunc() missing 1 required keyword-only argument: 'b'

I can then call it with “myfunc(5,6,7,8,9,b=22)” and we get:

'a=5, b=22, c=10, (6, 7, 8, 9)'

Positional-keyword separator

What if you want to have keyword-only parameters, but you don’t want the function to accept *args? Or what if you want your function to only accept keyword arguments (i.e., no positional arguments)? How can you do that?

If you put * in the list of parameters, then all parameters following it are keyword only. In other words, we can define:

def hello(*, first, last):
    return f'Hello, {first} {last}'

We can now call this function as “hello(first=’Reuven’, last=’Lerner’)”. But we cannot call it with any positional arguments.

You can, of course, put the “*” after one or more other parameters. Those preceding parameters can accept either positional or keyword arguments.

Positional-only parameters

The newest kind of parameter is the positional-only parameter. Such parameters, as you can gather from their names, cannot be set with keyword arguments. They are specified by putting a “/” (slash) in the parameter list. Any parameter that comes before a “/” is considered positional only. For example:

def add(a, b, /):
    return a + b

If I call it with “add(10, 3)”, then I get 13. But if I call it with “add(a=10, b=3)”, then I get the following error:

TypeError: add() got some positional-only arguments passed as keyword arguments: 'a, b'

I personally cannot think of where I would want to use positional-only parameters, but some builtin functions (e.g., “len”) use them, and they were added to Python so that you can do the same thing with functions written in Python as with functions written in C.

Summary

While Python doesn’t have parameter for different types of values, it does have different sorts of parameters, all of which tell Python whether they will accept positional arguments, keyword arguments, or both. Defining your parameters in the right way will make your code clearer, and make it easier for people who want to call your functions to know how they should do so.

Want to learn more? Check out my “Intro Python: Functions” and “Advanced Python Functions” courses, each of which includes hours of video instruction and dozens of exercises — the same material that I’ve been teaching to Fortune 500 companies for 20 years.

It’s time to learn Python

I’m on a mission: I want to help people to do more in less time, and to have better careers. How? Using Python.

If you’re reading this, then you might already be convinced that Python is both easy to learn and impressively powerful. And you hopefully benefit from this power, either in your day-to-day work or as you tinker on personal projects.

But a lot of people aren’t quite convinced, or don’t yet know:

  • Non-programmers think that programming is arcane, difficult to learn, or just irrelevant to their work. And besides, the way they currently do things is good enough, right?
  • Experienced developers, meanwhile, wonder why they should learn Python, when they can do just fine with other languages. I often hear this in my corporate training classes — until those hard-core developers see how much can be done with a handful of lines of Python.

I know, from experience, that no matter what your background, Python can help you to become more productive. And the number of companies looking for people with Python experienced continues to grow quickly, which means that having it on your resume gives you a leg up on other candidates.

On Wednesday, I’m holding a free, hour-long Webinar aimed at people who haven’t yet discovered the power and magic of Python. I’ll talk about why it’s a great language, and what people are doing with it. There will also be plenty of time for Q&A.

I hope to inspire people to start learning Python. And yes, I’ll also be pitching my courses as one way to make that move.

You can learn more, and register to join me, at https://FiveReasonsToLearnPython.com/.

Now, if you’re already an old hand at Python, then you probably don’t need convincing. But perhaps you have colleagues who aren’t sure if Python is a worthwhile investment of their time. Or maybe you have friends who have considered it, but aren’t completely sure.

And if you’re new to Python, like more than half of the people getting my free, weekly “Better developers” newsletter? Then you’ll almost certainly learn something new about the language, and ways to use it that you hadn’t considered before.

Questions? Contact me at reuven@lerner.co.il or as @reuvenmlerner on Twitter, and I’ll be happy to answer. But I also hope to see you on Wednesday!

Again, join me, for free, at https://FiveReasonsToLearnPython.com/.

Write better Python functions!

Weekly Python Exercise
Weekly Python Exercise

Whether you’re a newcomer to Python or an old hand, you’re probably writing lots of functions — functions that perform calculations, functions that parse files, functions that check passwords, and functions that contact remote APIs.

But in Python, functions are more than just verbs. They’re also nouns: They’re objects that we can store in data structures, and pass along to other functions as arguments.

Better yet, Python’s functions can be defined with a variety of parameters, to handle different types of arguments, passed in a variety of ways.

Learning how to use all of this functionality can be difficult. And understanding how to put it in context, wading through lots of documentation along the way, can seem daunting.

That’s where Weekly Python Exercise comes in. The upcoming cohort, which will start on Tuesday, April 27th, focuses on functions — how we define them, and how we use them. Over 15 weeks, you’ll learn how to write, call, and use functions with a variety of different parameters and in a wide variety of contexts.

Moreover, you won’t be doing it alone: You’ll be sharing your questions, answers, frustrations, and code with others in our private forum. And you’ll be invited to monthly office hours with me, to discuss any questions you might have about Python. Plus, the weekly exercises come with “pytest” tests, so that you can check your work and know when it fits the specifications.

If you want to improve your Python fluency, then there’s no better way than Weekly Python Exercise. Check out the A2 (functions for beginners) cohort at https://store.lerner.co.il/wpe-a2.

Questions? Thoughts? Eligible for a discount? Hit me up at reuven@lerner.co.il on e-mail, or as @reuvenmlerner on Twitter.

But don’t delay — because I won’t be offering this course again until 2022!

One year of “Python for non-programmers”

In the spring of last year, as the coronavirus pandemic began, it was pretty clear that this would be the major event of our lives, and that a lot of people were going to be affected in big, terrible ways — beyond the issues related to the virus itself, and the injury it created. Schools and businesses were shutting down, and we were all a bit scared about what was going to happen.

I decided to offer a free, weekly class, called “Python for non-programmers,” to whoever wanted to join. For 15 weeks, people from around the world joined me to learn Python programming. Whether they wanted to pick up a marketable skill, learn a new hobby, or just take their mind off of the pandemic, they were all welcome to join me. Many joined during the live sessions, but many others learned from the recorded videos.

A year later, I’m delighted to say that more than 2,500 people have taken this free course, and new people continue to sign up every day. I get e-mail about once a week from someone who said that they had always wanted to learn to program, but that other courses were too hard or too advanced, and they gave up. My course gave them the encouragement and support they needed.

It’s true that I earn my living teaching Python, and that I charge money for many of my courses. But it’s also true that I love what I do, in no small part because it helps people to improve their careers, and to accomplish more in less time. Knowing that so many people have taken this course is a bright spot in an otherwise difficult year. When I get e-mail from someone thanking me for the course, it completely makes my day, and reminds me why I so enjoy being a Python trainer.

If you have always wanted to program, but thinks that it’s too hard for them, I invite you to watch the 15 hours of video from “Python for non-programmers.” It is and will remain 100% free of charge.

And if you already know how to program, then please share the course info with people who don’t, but who might benefit from learning. Programming doesn’t have to be difficult or boring. Heck, it can even be fun, as well as useful to know.

Want to sign up? Just go to https://PythonForNonProgrammers.com/, and you’ll be able to start in no time.

Reminder: Weekly Python Exercise B1 (advanced topics 1) starts on Tuesday!

Weekly Python Exercise

If you’ve been using Python for a year or more, and want to sharpen your skills, then I have good news: A new cohort of Weekly Python Exercise starts this coming Tuesday.

The course has a simple formula, but one that works:

  • Every Tuesday, for the 15 weeks of the course, you get a question e-mailed to you on Tuesday morning, along with a description and “pytest” tests.
  • The following Monday, you’ll receive the solution, along with a detailed explanation.
  • In between (and after!) you can use our private forum to discuss the problem, and share your solution with others.
  • Once a month, I have live office hours, when I answer questions about the exercises (and Python in general).

Thousands of developers have improved their Python skills with WPE over the last few years. If you’re looking for something that won’t take too much time each week, and which will expose you to new ideas and Python development techniques, then join me in the coming cohort!

Want to join? Or to get a free sample? Or to learn more? Just go to https://WeeklyPythonExercise.com/!

As with all of my courses, I offer discounts to students, seniors/pensioners/retirees, people living outside of the 30 richest countries, and anyone affected adversely by the coronavirus pandemic. Just e-mail me at reuven@lerner.co.il with questions.

But don’t delay… the new cohort starts on Tuesday, and I won’t be offering B1 again until 2022.

12

You can, but should you? Combining some of Python’s more esoteric features

Note: As usual, readers of my blog (and my students) found a simpler and more elegant solution than I did… I knew that the boolean “or” operator returned the first “True” value it encountered, but I completely forgot that if it only encounters “False” values, it returns the final one. So there was a far more elegant solution than I suggested, namely “[LIST COMPREHENSION] or None”. I’ll leave this blog post up, since it was still fun to explore these ideas… but as usual, when you get too complex in Python, you’re probably overlooking a simpler and more straightforward solution, as I did here.

A few weeks ago, I held my monthly “office hours” session for subscribers to Weekly Python Exercise. WPE students are always invited not only to ask questions about what we’re learning in the course, but also any other Python-related issue that they have encountered.

Well, someone asked quite a doozy this month: He said that he wants to use a list comprehension to create a list for a project at work. Except that if the list comprehension is empty, then he wants to get a “None” value.

In other words: He wants a list comprehension, or at the very least an expression containing one, which will either return a list (if non-empty) or “None” (if the list is empty).

In answering this question, I managed to pull together what might be the greatest collection of unreadable Python constructs in a single expression. I’m not recommending that you write this sort of code — but it does demonstrate that Python’s syntax does lend itself to all sorts of creative solutions and possibilities, if you know how to combine things.

Let’s start by pointing out that a list comprehension always returns a list. Regardless of how many elements it might contain, the result of a list comprehension is always going to be a list. For example:

>>> [x*x for x in range(5)]
[0, 1, 4, 9, 16]

>>> [x*x for x in range(0)]
[]

In both of the above cases, a list value was returned; there’s no such thing as a list comprehension that returns a non-list value. Even an empty list is a list, after all.

My student would thus need to accept that while a list comprehension could be part of the solution, it couldn’t be the entire solution. We would need something like an if-else statement. For example:

mylist = [x*x for x in range(5)]

if mylist:
    output = mylist
else:
    output = None

The above code will certainly work, and would be my preferred way to solve such a problem. But for whatever reason, my student said that we needed to use a single expression; an if-else statement wouldn’t suffice.

Fortunately, Python does offer an inline, expression version of if-else. I personally find it hard to read and understand, but it’s designed for situations like this one, in which we need a conditional expression. It looks like this:

TRUE_OUTPUT if CONDITION else FALSE_OUTPUT

In other words, it’s a one-line “if-else” expression, returning one value if the condition is met, and another value if it is not. For example:

>>> 'Yes' if True else 'No'
'Yes'
>>> 'Yes' if False else 'No'
'No'

Of course, we can have any expression that we might like. So we could say:

>>> mylist = [x*x for x in range(5)]
>>> mylist if len(mylist) > 0 else None
[0, 1, 4, 9, 16]

In other words: If “mylist” is non-empty, then we’ll get “mylist” back. Otherwise, we’ll get “None” back. And it works!

However, it’s considered un-Pythonic to check for an empty list (or any other empty data structure) by checking its length. Rather, we can check to see if it’s empty simply by putting “mylist” in an “if” statement. In a boolean context, all lists (as well as strings, tuples, and dicts) return “True” so long as they contain any values, but “False” if they’re empty. We can thus rewrite the above code as:

>>> mylist = [x*x for x in range(5)]
>>> mylist if mylist else None
[0, 1, 4, 9, 16]

This is fine, but it’s not a single expression, which was a requirement. Fortunately, we can just squish everything into a single line, replacing any reference to “mylist” with the list comprehension itself:

>>> [x*x for x in range(5)] if [x*x for x in range(5)] else None
[0, 1, 4, 9, 16]

Now, this is getting pretty ugly. Among other things, we have repeated our list comprehension twice in the same line. After all, our one-line “if-else” expression is just that, an expression, with no assignment allowed. So if we want to keep things on a single line, only using expressions, there’s no way for us to store the output from our list comprehension for later, is there?

There wasn’t. But then came Python 3.8 with the “assignment expression” operator, aka “the walrus,” which changed everything. The walrus is designed to be used in just this kind of situation. OK, maybe not quite this ugly of an expression, but it can help us to get out of such pickles.

What I can do is use the walrus operator to capture the list created by the list comprehension. We can then use the variable to which we’ve assigned our list, thus saving us from having to use the list comprehension a second time.

Note that the one-line “if-else” is confusing on several fronts, but nowhere more so than the fact that the condition (in the middle of the expression) executes first, before either of the output expressions is evaluated. This makes sense, when you think about it. but it can still be confusing to put an assignment in the middle of a line, so that it can be used at the start of the line.

So, let’s try it:

output if output := [x*x for x in range(5)] else None

This doesn’t work, and that’s because we need to put the central expression inside of parentheses to ensure that Python’s parser knows what is going on:

>>> output if (output := [x*x for x in range(5)]) else None
[0, 1, 4, 9, 16]

It worked! Thanks to a combination of the condition expression, the walrus operator, list comprehensions, and the fact that empty lists are “False” in a boolean context, we managed to get a single expression that returns the result of a list comprehension when it contains values, and “None” otherwise.

And while I would question the wisdom of having such code in an actual production system, I freely admit that there are times when such hacks are necessary. And despite the fact that Python has a more rigid syntax than many other languages, its functional parts made it possible for us to achieve our goal with only a minimum of code.

1 2 3 24
>