PyCon didn’t happen in Pittsburgh, as planned, thanks to the coronavirus and covid-19. But it did happen online, and I was delighted to be able to present a talk!

Here’s the talk video:

And here are the slides, which you can download and use:

Please send comments via e-mail to reuven@lerner.co.il or on Twitter to @reuvenmlerner!

It happens to all of us: You write some Python code, but you encounter an error:

>>> for i in 10:    # this will not work!
        print(i)

TypeError: 'int' object is not iterable

This isn’t just an error, but an exception. It’s Python’s way of saying that there was a problem in a defined way, such that we can trap it using the “try” and “except” keywords.

Just like everything else in Python, an exception is an object. This means that an exception has a class — and it’s that class we use to trap the exception:

try:
    for i in 10:
        print(i)
except TypeError as e:
    print(f'Problem with your "for" loop: {e}')

We can even have several “except” clauses, each of which looks for a different type of error. But every Python class (except for “object”) inherits from some other class, and that’s true for exception classes, as well. So if we want to trap both “KeyError” and “IndexError”, then we could name them both explicitly. Or we could just trap “LookupError”, the parent class of both “KeyError” and “IndexError”.

Python’s exception-class hierarchy is visible at https://docs.python.org/3/library/exceptions.html#exception-hierarchy, in the documentation for the Python standard library. It’s useful for knowing what exceptions exist in Python, how the hierarchy looks, and for generally understanding how exceptions work.

But if you look at the bottom of that hierarchy, you’ll see that there’s an exception class called “Warning,” along with a bunch of subclasses such as “DeprecationWarning” and “BytesWarning”. What are these? While they’re included along with the exception hierarchy, warnings are exceptions, but they’re neither raised nor used like normal exceptions. What are they, and how do we use them?

First, some history: Warnings have been around in Python for quite some time, since Python 2.1 (back in the year 2000). PEP 230 was written by Guido van Rossum (the original author of Python and the long-time BDFL), and was added not only to create a mechanism for alerting users to possible problems, but also for sending such alerts from within programs and for deciding what to do with them.

Why warnings?

Before I show you how you can use warnings in your own code, let’s first consider why and when you would want to use warnings. After all, you could always use “print” to display a warning. Or if something is really wrong, then you could raise an exception.

But that’s just the point: There are times when you want to get the user’s attention, but without stopping the program or forcing a try-except clause. And while “print” is often useful, it normally writes to standard output (aka “sys.stdout” in Python), which means that your warnings could get mixed up with the warnings themselves.

(And while I just wrote that you might want to get the user’s attention, I’d argue that most of the time, warnings are aimed at developers rather than users. Warnings in Python are sort of like the “service needed” light on a car; the user might know that something is wrong, but only a qualified repairperson will know what to do. Developers should avoid showing warnings to end users.)

You can also imagine a situation in which some warnings are more important than others. You could certainly devise a scheme in which the program would “print” warnings, and that the warning’s first character would indicate its severity… but why work in this way, when Python has a complete object system, as well as complex data types at its disposal?

Moreover, there are some situations in which a user might not want to dismiss warnings. Maybe I’m running a very sensitive type of program in production, and I’d rather have the program exit prematurely than continue in a potentially ambiguous situation.

Python’s warning system takes all of this into account:

• It treats the warnings as a separate type of output, so that we cannot confuse it with either exceptions or the program’s printed text,
• It lets us indicate what kind of warning we’re sending the user,
• It lets the user indicate what should happen with different types of warnings, with some causing fatal errors, others displaying their messages on the screen, and still others being ignored,
• It lets programmers develop their own, new kinds of warnings.

Not every program needs to have or use warnings. But your program would like to scold a user for the way that a module was loaded or a function was invoked, then Python’s warnings system provides just what you want.

Warning the user

Let’s say that you want to warn the user about something. You can do so by importing the “warnings” module, and then by using “warnings.warn” to tell them what’s wrong:

import warnings

print('Hello')
warnings.warn('I am a warning!')
print('Goodbye')

What happens when I run the above code (in a file I called “warnings1.py”? The following:

Hello
./warnings1.py:6: UserWarning: I am a warning!
warnings.warn('I am a warning!')
Goodbye

In other words, the above was all written to my terminal screen. The three lines were all printed in sequence, so it’s not as if the warning was printed at a separate phase (e.g., compilation) of the program. But there is a clear difference between the plain ol’ “print” statements and the output I got from the warning.

First of all, we’re told in which file, and on which line, the warning took place. In a tiny and trivial example like this one, that seems like overkill. But if you have a large application consisting of many different files, then it’ll certainly be nice to know what code generated the warning.

We’re also told that this was a “UserWarning” — one of the types of warnings that we can generate. Just as different types of exceptions allow us to selectively trap them, different types of warnings allow us to handle them differently.

But there’s also something hidden from this output: The “print” statements and my “warnings.warn” statement actually sent their output to two different places. As I wrote above, “print” normally writes to “standard output,” aka “sys.stdout”, typically connected to the user’s terminal window. But “warnings.warn” normally writes to “standard error,” aka “sys.stderr”. The problem is that by default, “sys.stdout” and “sys.stderr” both write to the same place, namely the user’s terminal.

But look at what happens if I redirect program output to a file:

$ ./warnings1.py > output.txt

./warnings1.py:6: UserWarning: I am a warning!
warnings.warn('I am a warning!')

I told my Unix shell that I wanted to run “warnings1.py”, and that all output should be placed in “output.txt”, rather than displayed on the screen. But I didn’t really say “all output.” Rather, by using the “>”, I only redirected output sent to “sys.stdout”. Warnings, which are sent to “sys.stderr”, are still displayed. This is normally considered to be a good thing, ensuring that even if you’ve redirected output to a file, you’ll still be able to see warnings and other errors. So while sys.stdout and sys.stderr both go to the same destination by default, we can see the advantage of being able to separate them.

Different types of warnings

Let’s say that I’m maintaining a library that has been around for some time. The library has a function that works, but which is a bit old-fashioned, and doesn’t support modern use cases. It’s a pain for me, as the library maintainer, to support two versions of the function — the old one and the new one.

I can declare, in documentation and social media, that a new version (3.0) of my library will be out next year, and that this new version won’t support the old version of the function. But we all know that programmers don’t tend to read documentation. So I would prefer to shock users a bit, telling them that while the old function version still works, they should start to move to the newer version.

How can I do that? With warnings, of course! Here’s an example of how that might look:

import warnings

def hello(name):
    warnings.warn('"hello" will be removed in version 3.0')
    return f'Hello, {name}!'

def newhello(name, decoration=''):
    return f'Hello, {decoration}{name}{decoration}!'

print(hello('world'))
print(newhello('world', decoration='*'))

Now, any time a user runs the function “hello”, they’ll get a warning. Moreover, because this warnings goes to standard error (and not standard out), it won’t be mixed with the normal output. Here’s the output from the above:

$ ./warnings2.py

./warnings2.py:7: UserWarning: "hello" will be removed in version 3.0
warnings.warn('"hello" will be removed in version 3.0')
Hello, world!
Hello, *world*!

But it gets better than that: Maybe we want to separate our normal, run-of-the-mill warnings from other types of warnings. For example, we might have a number of functions that are deprecated. To handle this, the “warnings.warn” function supports an optional, second argument — a category of warning. For example, we can use DeprecationWarning:

import warnings

def hello(name):
    warnings.warn('"hello" will be removed in version 3.0',
                   DeprecationWarning)
    return f'Hello, {name}!'

def newhello(name, decoration=''):
    return f'Hello, {decoration}{name}{decoration}!'

print(hello('world'))
print(newhello('world', decoration='*'))

We don’t have to “import” DeprecationWarning, or any other of the standard warning types, because they’re already imported automatically, into the “builtins” namespace that’s always available to a Python program. And indeed, there are a number of such warning classes that we can use, including UserWarning (the default), DeprecationWarning (which we used here), SyntaxWarning, and UnicodeWarning. You can use whichever one of these you deem most appropriate.

You might have noticed that these warning categories are precisely the same classes as we saw earlier, when we were looking through Python’s built-in exception hierarchy. And indeed, this is how those classes are meant to be used, passed as a second argument to “warnings.warn”.

Simple filtering

Let’s say that you are using a bunch of old functions, and that each of those functions are going to warn you that you should really switch to their newer alternatives. You’ll probably get a bit annoyed if every time you run your program, you get a bunch of warnings. The warnings are there to inform you that you should upgrade… but sometimes, the warnings are more annoying than helpful.

In such a case, you might want to filter out some of the warnings. Now, “filtering” is a very general term that’s used by the warnings system. It basically lets you say, “When a warning that matches certain criteria fires, do X with it” — where X can be a variety of things.

The simplest filter is “warnings.simplefilter”, and the simplest way to invoke it is with a single string argument. That argument tells the warning system what to do if it encounters a warning:

• “default” — display a warning the first time it is encountered
• “error” — turn the warning into an exception
• “ignore” — ignore the warning
• “always” — always show the warning, even if it was displayed before
• “module” — show the warning once per module
• “once” — show the warning only once, throughout the program

For example, if I want to ignore all warnings, I could say:

warnings.simplefilter('ignore')

And then if I have code that reads:

print('Hello')
warnings.warn('The end is nigh!')
print('Goodbye')

We’ll see output that looks like:

Hello
Goodbye

As you can see, the warning disappeared entirely, thanks to the use of “ignore”.

What happens if we take the other extreme, namely we turn the warnings into exceptions?

warnings.simplefilter('error')
warnings.warn('Yikes!')

Sure enough, we then get an exception:

UserWarning: Yikes!

As you can see, we got a UserWarning exception. We can use “try” and “except” on these, trapping them if we want… although I must admit that it seems weird to me to turn warnings into exceptions, only to trap them. (I’m sure that there is a use case for this, though.)

More specific filtering

I mentioned that “simplefilter” takes a mandatory argument, and we’ve seen what those can be. But it turns out that “simplefilter” takes several additional, optional arguments that can be used to specify what happens when a warning is issued.

For example, let’s say I want to ignore UserWarning but turn DeprecationWarning into an exception. I can say:

import warnings

warnings.simplefilter('ignore', UserWarning)
warnings.simplefilter('error', DeprecationWarning)

warnings.warn('bad news!')  # ignored

warnings.warn('very bad news', category=DeprecationWarning)

This code results in the following output:

Traceback (most recent call last):
File "", line 1, in
DeprecationWarning: very bad news

In other words, we successfully ignored one type of warning, while turning another into an exception — which, like all exceptions, is fatal if ignored.

The “simplefilter” function takes four arguments, all but the first being optional

What else can you do?

The warning system handles a very wide variety of cases, and can be configured in numerous ways. Among other things, you can:

• Define warning filters from the command line, using the -W flag
• Set multiple filters, each handling a different case
• Specify the message and module that should be filtered, either as a string or as a regular expression
• Create your own warnings, as subclasses of existing warning classes
• Capture warnings with Python’s logging module, rather than printing the output to sys.stderr.
• Have output go to a callable (i.e., function or class) of your choice, rather than to sys.stderr, for fancier processing.

Interested in learning more?

• The original document about warnings, PEP 230, is a good starting place that also describes the motivations.
• The documentation for the “warnings” module in the standard library describes everything I wrote here, and quite a lot more.
• The “Python Module of the Week” site has as great introduction here: https://pymotw.com/3/warnings/

If you’ve programmed in Python for even a short amount of time, then you’ve probably written a fair number of functions.

But many newcomers to Python don’t understand just how useful and powerful functions can be:

• We can treat functions as nouns, not just as verbs — passing them as arguments, and storing them in variables
• We can take define many different types of parameters, each with its own semantics and advantages
• We can use functions written by other people, in external modules — those that come with Python’s standard library, and those we download from PyPI

These techniques aren’t just interesting. They can help you to write better, larger, and more sophisticated Python applications.

If you’re looking for a better Python job — in machine learning, Web development, analytics, or devops — then this will certainly help you to improve your understanding and fluency.

I’m starting a new cohort of Weekly Python Exercise on May 5th, one that’s all about functions and modules, aimed at beginners with Python — those with less than one year of experience with the language. Over 15 weeks, you’ll become a more fluent Python programmer, doing more with less code and becoming more confident in what you’re doing.

The course, like all WPE cohorts, has a simple formula:

• On Tuesday, you get a new question, along with “pytest” tests
• On the following Monday, you get the solution and a full explanation
• In between, you chat with others in our private forum, and discuss possible answers
• Once per month, I hold live office hours, where we can discuss questions you might have about the exercises — or any other Python questions you have.

WPE is now in its fourth year, with many hundreds of satisfied students. I’m confident that if you’ve been using Python for less than a year, this cohort of WPE will help you to improve your knowledge of functions.

Join Weekly Python Exercise A2: Functions for beginners, starting on May 5th. Get a better job — or just do your current job better.

Questions or comments? E-mail me at reuven@lerner.co.il, or on Twitter as @reuvenmlerner.

And don’t forget that I give discounts to (1) students, (2) seniors/pensioners/retirees, (3) anyone living outside of the world’s 30 richest countries, and (4) anyone affected adversely by the coronavirus/covid-19. Just e-mail me at reuven@lerner.co.il if any of these applies to you.

My free, weekly “Python for non-programmers” course continues tomorrow, Friday April 24th, at 10 a.m. Eastern.

If you’ve ever wanted to learn to program, then you’re always free to join. (About 1,600 people have already done so.)

And yes, we’ve been going for a month already… but if you join, you have access to all of the previous recordings, as well as our private forum. So you can always catch up!

Learn more, and join (for free!) at https://PythonForNonProgrammers.com/ .

Questions or comments? Just e-mail me at reuven@lerner.co.il.

My free, weekly “Python for non-programmers” course continues! Today, we’ll start to talk about “strings” — in other words, text! Letters, words, sentences, and anything else contains text is a string in Python.

Join me for the live session, and get access to all previous recordings, Jupyter notebooks, and our private forum, by registering at https://PythonForNonProgrammers.com/. It’s totally free of charge to join.

If you’ve always wanted to learn to program, then this is your chance. I hope that you’ll join me and hundreds of others in learning Python.

I look forward to seeing you in the course! Any questions? Just contact me at reuven@lerner.co.il or on Twitter as @reuvenmlerner.

If you’ve never programmed a computer before — or if you tried, and found it frustrating and difficult — then you’re welcome to join my “Python for non-programmers” course, which takes place on Thursdays at 10 a.m. Eastern.

The class is 100% free of charge, and open to anyone who wants. Just register at https://PythonForNonProgrammers.com/. Registering gets you weekly reminders, recordings of previous sessions, and invites to the private forum, where you can chat about the lessons with other students.

And it’s a live class, so you can ask questions during the session, and get an immediate answer from me.

(So yes, this means that if you’re only joining now, you can catch up. And if you miss this or any other lesson, you can access the recording down the line.)

Stay home and stay safe — but while you’re home, take advantage of the opportunity to learn Python! I guarantee that you’ll enjoy it. (And at this price, why not give it a whirl?)

Once again, you can register at https://PythonForNonProgrammers.com/.

This is just a reminder that my 100% free, weekly “Python for non-programmers” course is continuing tomorrow (Friday, March 27th) at 10 am Eastern. Join me for live discussion, coding, and Q&A as we march (slowly) through the Python language.

Sounds interesting? Sign up at https://PythonForNonProgrammers.com !

We’ll be talking about making decisions with the “if” statement — one of the most common, but also important, parts of programming!

Signing up gives you reminders before each session, access to previous recordings, and access to our exclusive forum, where you’ll get homework questions, and be a part of our growing community of people who want to learn Python.

I hope to see you there. Questions or comments? Just contact me at reuven@lerner.co.il, or on Twitter as @reuvenmlerner

If you’ve been using Python for a while, and want to up your game, then I’ve got great news for you: I have released three new advanced Python courses, all available immediately from my online store:

Advanced Python data structures

This 5-hour course assumes that you’re familiar and comfortable with strings, lists, tuples, and dicts, and takes you to the next level. First, we review the built-in data structures, going deeper into understanding how they work, their relative efficiencies, and some more advanced techniques for working with them. Then we look at combinations of these data structures, and the trade-offs associated with such multi-level collections. Finally, we look at some the interesting data structures provided in Python’s standard library. Available at https://store.lerner.co.il/advanced-python-data-structures

Advanced Python functions

This 3.5-hour course assumes that you are familiar and comfortable with the basic writing and usage of functions, and want to go deeper. First, we talk about function objects and byte codes, and how Python creates them. Then we talk about Python’s various scopes, including nested functions — and some of the ways in which they can be useful. Finally, we talk about such topics as stack frames and type annotations. Available at https://store.lerner.co.il/advanced-python-functions

Advanced Python objects

Finally, this 7-hour course discusses Python objects in great depth. First, I introduce a number of the “magic” methods that you can implement on your objects.  Then we discuss such topics as context managers, multiple inheritance, properties, descriptors, and even metaclasses. If you’ve always wanted to learn about how Python objects work, and how you can use them to your advantage, this course is for you. Available at https://store.lerner.co.il/advanced-python-objects .

Python for non-programmers

is my 100% free, live, weekly course. Every Friday at 10 a.m. Eastern, I introduce you to the world of programming using Python. Anyone who signs up (for free) gets invited to the weekly Zoom session, as well as access to all of the videos (as they’re recorded) and Jupyter notebooks I use when teaching. If you’re new to programming, then I invite you to join me — and if you have friends, family, or colleagues who might be interested, please tell them! SIgnup is at https://PythonForNonProgrammers.com/ .

Affected by the coronavirus?

I’ve long tried to make my courses as affordable as possible, giving discounts to students, seniors/pensioners/retirees, and people who live outside of the world’s 30 richest countries.

Today, I’m announcing a new discount code, for people whose job and/or finances have been disrupted by the coronavirus, covid-19, and the many types of pain that it is generating.  If you have been affected adversely by this crisis, and are interested in purchasing my courses, please e-mail me, at reuven@lerner.co.il. (You should also contact me if you’re eligible for any of the other discounts.) I’ll reply with an appropriate coupon code.

Questions?

Just contact me via e-mail (reuven@lerner.co.il) or as @reuvenmlerner on Twitter!

This is a tough time for the world. Wherever you live, you have likely been affected by covid-19, the coronavirus that has been making its way to every country, city, and town.

Many countries, companies, and individuals are now restricted to their homes.  This can be frustrating in many ways.  Moreover, I’m not alone in believing that we’re about to see some very troubled times for the world economy.

I’ve been trying to decide what I can do, as a Python instructor, to help people in these trying times.  And after some thought, I’ve decided to offer a free, weekly live workshop for people with little or no Python programming experience. 

This workshop will be run as a live Zoom session on Fridays, with me teaching Python programming from the ground up.  This is similar to the “Python for non-programmers” course that I’ve been giving to Fortune 500 companies for the last few years — although it’ll be broken up over many weeks, and will hopefully have many more participants than I’ve ever had before.

This is an experiment, and I’m asking for your help in letting people know about it. If you, or someone you know, wants to spend an hour a week learning programming basics, then I invite you to join me in my “Python for non-programmers” workshop. 

I’ll ask questions, I’ll give exercises, and I’ll take questions. And I’ll tell lots of bad jokes, too. But in the end, I hope that you’ll learn, gain some skills, and have some fun not thinking about the news.

Not only will this workshop be completely free of charge, but I’ll be sharing each week’s recordings, as well. So if you cannot attend, or if you want to catch up on old videos, or even check out the Jupyter notebooks I use in each class, then don’t worry — all of that will be available to you, for as long as you want.

Sounds good?  I sure hope so!  You can join here:

Join my free, weekly “Python for non-programmers” workshop

When you join, I’ll send a welcome message.  And then I’ll send you connection instructions on Wednesday or Thursday.

Questions? Just contact me, via e-mail (reuven@lerner.co.il) or on Twitter (@reuvenmlerner).

If you’ve been using Python for a while, then you know that it’s easy to learn — and even fun to use. But you probably also find yourself stuck on occasion, frustrated that the ideas and techniques don’t flow more easily.

The only solution to that problem is practice. And in Weekly Python Exercise, you get that practice — 15 weeks of exercises, accompanied by “pytest” tests, a private discussion forum, and monthly office hours.

The B1 (advanced topics, part 1) cohort starts tomorrow, March 10th. If you want to improve your skills with data structures, functional programming, classes, generators, decorators, and threads, then this is for you.

But don’t delay, because this is the last time I’ll be offering B1 in 2020.

Learn more, and sign up, at the Weekly Python Exercise B1 home page.