Archive

Category Archives for "Python"
1

Looking for Python podcast co-hosts

As you might know, I’m a panelist on the weekly “Freelancers Show” podcast, which talks about the business of freelancing.

The good news: The same company that’s behind the Freelancers Show, Devchat.tv, is putting together a weekly podcast about Python, and I’m going to be on that, too! We’ll have a combination of discussion, interviews with interesting people in the Python community, and (friendly) debates over the current and future state of the language.

The better news: We’re looking for cohosts to participate in our panel discussion on a regular basis, say 2-4 times per month.

We’re looking for a diverse set of hosts, representing the breadth and width (and height, I guess) of the Python community — including skill levels, technical backgrounds, workplaces, and interests. You’ll also need a decent podcasting microphone. We’ll record weekly, on a day to be determined, at about 20:00 UTC for about 60-90 minutes.

If you’re talkative, articulate in English, interested in Python, able to commit to recording several times per month, and are willing to tolerate my jokes, then please be in touch, via e-mail (reuven@lerner.co.il) or Twitter (@reuvenmlerner)! We’re hoping to start in the coming month or two.

I should note that this podcast is not meant to replace the existing, and amazing, podcasts that already exist in the Python community! I listen to them (and occasionally appear on them, too), and have a lot of respect for the hosts of (for example) Talk Python to Me, Python Bytes, Python Init, and Test & Code. I hope that what we create will be complementary to their shows, and grow the Python community to an even larger degree.

Sound interesting? Let me know!

1

Last change to join Weekly Python Exercise: Beginner objects

If you have been using Python, but don’t quite understand how and when to write and use the language’s object-oriented facilities, then I have good news and bad news:

  • Good news: The new cohort of WPE: Beginner objects starts tomorrow, and includes 15 weekly exercises about classes, objects, instances, methods, attributes, and other Python object goodies.
  • Bad news: The new cohort of WPE: Beginner objects starts tomorrow, which means that you only have until TONIGHT to join the new cohort.
  • Even worse news: I’ll only be running this version of WPE in about a year, toward the end of 2020. So if you don’t join this cohort, then you’ll have to wait quite some time.

Here’s what people have previously said about Weekly Python Exercise:

  • “What differentiates Reuven’s course from regular online reading, YouTube videos, or other self study mechanisms are context and application.”
  • “It’s totally worth it and a fantastic way to keep my Python skills sharp and learn new things in the process.”
  • “Fully recommend the course to anyone wanting to not only begin with Python, but learn it contextually and apply the learning via best practices.”

Remember, if you’re dissatisfied, then I offer a 100%, no questions asked, refund. 

So, don’t wait: Read more about Weekly Python Exercise (at https://WeeklyPythonExercise.com/). Questions or comments? Just e-mail me at reuven@lerner.co.il, or contact me on Twitter as @reuvenmlerner. I’ll answer your query right away!

7

LEGB? Meet ICPO, Python’s search strategy for attributes

When it comes to variables, Python has a well-known search strategy, known by the acronym “LEGB.” Whenever you mention a variable — and by “variable,” I mean a name that could be referencing data, a function, or a class — Python tries to find it in four different places: The local (function) scope, the enclosing function’s scope, the global scope, and finally in the “builtins” namespace.

Variable scoping seems both boring and annoying, but it actually explains a lot about Python’s design. It’s really worth learning about, and won’t take much of your time. Indeed, I have a free e-mail course on the subject; you’re welcome to subscribe.

But what about attributes? How does Python search for those?

(Quick aside: If you’re wondering what “attributes” are, then consider that when you say “a.b” in Python, the “a” is a variable and the “b” is the attribute “b” on “a”, not a variable. As a general rule, anything with a dot before its name is an attribute. And yes, there are exceptions to this rule, but it’s a good generalization.)

From my experience, this question seems like an odd one to many Python developers, including those who have been using the language for a while. What does it mean to “search for attributes”? Aren’t attributes attached to an object, in a (sort of) private dictionary?

The answer, of course, is both “yes” and “no.” Attributes do belong to a single object. But in many cases, when Python cannot find an attribute on one object, it’ll look on another object to find it.

Indeed, this search for attributes sits at the heart of the Python language, and explains many of the things that we’ve come to expect, such as method calls.

I’ll start from the end: Just as the acronym LEGB (local, enclosing, global, builtins) makes it easy (or easier) for us to understand and follow Python variable lookups, I use the acronym ICPO (instance, class, parent, object) to understand where Python searches for attributes. Keeping this search path in mind will help you to both read and write Python more easily.

I’ll expand the acronym here, and then go through a bunch of examples, so that you can understand it better:

  • Instance: When we ask for a.b, Python first checks: Does the object “a” have an attribute “b”? If so, then “b” is retrieved from “a”, and the search ends.
  • Class: If Python doesn’t find “b” on object “a”, then it looks for type(a).b. That is, it looks for an attribute “b” on a’s class. If it finds the attribute here, then it returns the value, and the search ends.
  • Parents: If Python doesn’t find “b” on type(a), then it looks on the parents of type(a):
    • If the class inherits directly from “object”, then there aren’t really any parents from which to inherit, and this phase is skipped.
    • If the class inherits from one class, then we check there — and on its parents, and its parents, etc.
    • If the class inherits from multiple classes, then we follow the MRO (method resolution order) of the class.
  • Object: All classes in Python inherit, directly or indirectly, from “object”, the top of our class hierarchy. As a result, searching for an attribute, if not first found elsewhere always concludes on “object”. If we cannot find an attribute on “object”, and it wasn’t found elsewhere previously, then Python raises an AttributeError exception.

I is for “instance”

Let’s start with some simple code to understand what I mean by all of this:

class Foo():
    def __init__(self, x):
        self.x = x

f = Foo(10)
print(f.x)  

In the above code, we define a class, Foo, with a single method, __init__. (We’ll return to methods in a little bit.) When we create a new instance of Foo, aka “f”, we create a new attribute on “self”, the instance.

Take note of this: Whenever we add or update an attribute on “self”, we’re doing so on the instance. In this case, it’s an instance of Foo, not the Foo class. Just as there’s a difference between an auto factory and an individual car, so too is there is a difference between the class Foo and f — and in a method, “self” points to the individual instance.

Thus, when we ask (on the final line) to see the value of “f.x”, Python goes through its ICPO search path, first asking: Is “x” an attribute on the instance, which we call “f”? Happily, the answer is “yes,” the search ends, and we get our answer of 10 returned to us.

C is for “class”

The above is probably how most people think of attributes, and attribute lookups, in Python. But things are a bit more complex than that. Let’s make our class a tiny bit more interesting:

class Foo():
    def __init__(self, x):
        self.x = x

    def x2(self):
        return self.x * 2  

f = Foo(10)
print(f.x2())  

Once again, we’ve defined a class Foo. And once again, the __init__ method will define an attribute named “x” on our new instance.

But this time around, we’re not asking for “f.x”. Rather, we’re going to execute the method “f.x2”. So Python starts off asking if the object “f” has an attribute named “x2”.

Except that the answer is “no,” because methods are class attributes. “x2” doesn’t exist on “f”; “x2” was defined on the class “Foo”. And thus, when Python cannot find “x2” on “f”, it goes to the next stop on its search, namely on f’s class — Foo.

Does Foo have an attribute “x2”? It does, a method object. That object is returned, and then the parentheses tell Python to execute it. When the method is executed, it does Python’s magic switcheroo, turning “f.x2()” into “Foo.x2(f)”, thus passing an argument to the “self” parameter. And the method runs!

What happens if I define a new attribute on “f” whose value is “x2”? By the ICPO rule, that attribute would have priority, and would effectively make calling the method impossible via the instance.

NOTE: This is not something you would normally want to do.

Let’s throw caution and sanity to the wind, and try it:

class Foo():
    def __init__(self, x):
        self.x = x

    def x2(self):
        return self.x * 2  

f = Foo(10)
f.x2 = lambda: 'Not the x2 you expected'
print(f.x2())  

When we run the above code, Python first checks on “f”, to see if it has an attribute “x2”. And the answer is “yes.” It stops searching, and returns the function that we defined with “lambda”. The function is then executed via the parentheses, and returns a string value.

Note that this hijacked version of “x2” is only available on “f”. If we were to define another, separate instance of “Foo”, on which we didn’t define an “x2” attribute, the ICPO rule would fail to find “x2” on the instance, which means it would then search on Foo (i.e., the class). Sure enough, there’s an “x2” attribute on the class “Foo”, which would be returned.

P is for “parent”

What happens, though, if the attribute isn’t found either on the instance or on the class? Let’s look at an example:

class Foo():
    def __init__(self, x):
        self.x = x
    def x2(self):
        return self.x * 2

class Bar(Foo):
    pass

b = Bar(10)
print(b.x2())

When we create b, our new instance of Bar, Python looks for __init__ on it. But there is no attribute “__init__” defined on b. So it looks on b’s class, Bar. The attribute isn’t there, either. Python then falls back to its third ICPO possibility, the parent. Bar only inherits from a single class, Foo. Sure enough, Foo.__init__ does exist — so that method attribute is retrieved, and then executes.

In other words: The ICPO rule describes how inheritance functions in Python. If we don’t find a method in a class, we look in its parent class. But inheritance works on all attributes, not just methods; it’ll work for data, as well. That’s why, if you create a class attribute, it’s available via the instances:

class Foo():
    y = 100

    def __init__(self): 
        pass

f = Foo()
print(Foo.y)
print(f.y)

When we run the above code, we see “100” printed twice: The first time, because Python asks if “y” is an attribute on Foo, and the answer is “yes.”

Wait a second: Isn’t “Foo” a class? Why is it getting searched first?

Because in Python, classes are indeed special. But they’re also objects like everything else in the language. So yes, “f” is an instance of “Foo”. But “Foo” is an instance of “type”. If we ask for “Foo.y” and the attribute “y” doesn’t exist, then the ICPO rule tells Python to look at type(Foo), which is “type”. Fortunately, in this case, that doesn’t happen, and we get the value “100” back.

Then, we ask for “f.y”; this also returns “100” — because by the ICPO rule, Python looks on the instance “f”, fails to find attribute “y”, and then goes to the class “Foo”, where it finds the (class) attribute “y”.

As always with ICPO, the first match that Python finds wins. This means that if a subclass and a parent class both have a method of the same name, the subclass’s method will execute:

 class Foo():
     def __init__(self, x):
         self.x = x

     def x2(self): 
         return self.x * 2

 class Bar(Foo):
     def x2(self):
         return self.x * 22

 b = Bar(10)
 print(b.x2())  # prints 220

In the above example, we invoke “b.x2()”:

  • Python looks for “b.x2”, and doesn’t find it.
  • Python looks for “Bar.x2”, finds it, and executes it.
  • Python does not continue searching, and thus “Foo.x2” is not executed. Which is precisely what we want, but is (in my experience) surprising to many newcomers to Python.

Find, but what about multiple inheritance? In such a case, the “P” stands for parents (plural) and not parent (singular). Python will search through each parent class, one at a time, according to the MRO (method resolution order). For example:

class A():

    def __init__(self, x):
        self.x = x
    def x2(self):
        return self.x * 2

class B():

    def __init__(self, y):
        self.y = y
    def y2(self):
        return self.y * 2

class C(A, B):
      pass

In the above code, class C inherits from both A and B. If we ask for its MRO, we’ll find that it first looks on itself (as usual), then A, then B, and then (finally) object. This means that if we say:

c = C(10)
print(vars(c))

The above will show that c has a single attribute, “x”, whose value is 10. That’s because when we created “c”, Python looked for c.__init__. It kept following the ICPO rule until it found the first parent class, A, where __init__ was defined. That method ran, but B’s __init__ didn’t. This means that “c” doesn’t have the “y” attribute. And thus, if we write:

print(c.y2())

We’ll get an error — not an indication that the method is missing, because the “y2” method is indeed found, by searching “c”, then the class “C”, then its parent “A”, and then finally its parent “B”. And indeed, we find the “y2” method there!

So what’s the problem? The “y2” method expects the object (self, aka our instance “c”) to have an attribute “y”. But because only A.__init__ ran, and not B.__init__, there is no “y” attribute, and we get an error.

The error is obviously not something we want, but it is the natural result of the ICPO rule.

O is for “object”

The final location in which Python looks for an attribute is “object,” the top of our object hierarchy. In Python 2, you had to explicitly inherit from “object” to avoid having an old-style class, which worked just like modern classes… until it didn’t. Nowadays, we don’t have to worry about this; all classes automatically inherit from “object”, whether you state this expressly or not.

“object” doesn’t actually have a lot defined on it. There are some methods that are used as defaults, such as “__init__” (which does nothing, and fires if you don’t provide an “__init__” method on your class) and “__str__” (which ensures that all objects can be cast into strings). In many cases, you’ll want to implement — and thus override — these default methods, so that your objects can be initialized appropriately, as well as be cast as strings in the right way.

Why should I care?

You’ve read pretty far down in a long blog post — so I hope that you care! But if you got to this point and aren’t yet convinced of the importance of this rule, consider:

  • Every time you invoke a method, Python uses the ICPO rule to find it. In other words: Inheritance in Python is a directly outgrowth of the ICPO rule.
  • The fact that everything in Python, including classes, are objects, means that everything follows the same rules. The ICPO rule applies to your instances and classes, but also to built-in instances and classes. Python is nothing if not consistent, and your objects fit into this consistent hierarchy.
  • It’s common for people coming from other languages to talk about “instance variables” and “class variables.” Abandoning those terms in favor of “attributes,” along with the ICPO rule, will help you to understand how Python works, and why it is (or isn’t) finding the data you asked for.

At the end of the day, the entire object system in Python boils down to a few rules and systems. An important part of this system is the ICPO rule. Once you’ve internalized it, many things that previously seemed odd about Python will (I believe) be more straightforward and consistent.

So, let me know: Does this make things easier to understand? What does this not explain? Leave a comment here, and I’ll try to respond and/or update the article!

1

Learn and understand Python objects — with Weekly Python Exercise, starting September 17

I’ve found that many Python developers are confused by object-oriented programming in the language:

  • Newcomers to programming are confused by object-oriented ideas and terminology, while
  • Veteran programmers are surprised and confused by how Python objects work

No matter which camp you’re in, I have good news for you: Weekly Python Exercise will be starting a new cohort on September 17th, and it’s all about object-oriented programming in Python. Join me as we learn by doing, solving problems with objects, instances, classes, attributes, class attributes, and inheritance.

By the end of the course, you’ll understand how objects work, and will know how to use them in your own projects.

Join me and other Python developers from around the world in this upcoming cohort. To sign up, see samples, or learn more, just go to https://WeeklyPythonExercise.com/ . See you on the 17th!

1

“Python Workout” is today’s Manning “Deal of the day”

Want to improve your Python fluency? There’s no better way than doing exercises, practicing your Python skills and ensuring you understand how to solve problems in the best possible way.

My book, “Python Workout,” not only contains 50 of my favorite exercises from my Python training business (at some of the world’s best-known companies), but is on sale today as Manning’s “Deal of the Day,” together with two other Python related books.

Don’t miss this opportunity to improve your Python skills at a discount! Go to https://www.manning.com/dotd to take advantage of the sale price.

2

Reminder: Only one day left for early-bird pricing on Weekly Python Exercise A3 (objects for beginners)

The biggest problem with software today isn’t writing code. It’s maintaining — debugging, improving, and expanding — existing code. It’s hard to maintain someone else’s code. Heck, it’s even hard to maintain your own code. (Who among us hasn’t looked at code and said, “Who was the idiot who wrote this… oh, it was me…”?)

There’s no magic formula that’ll make code maintenance easy. But you can make it easier if (1) everyone agrees on some conventions for how the code will look and act, and (2) if you can reuse existing code, and thus write less of it.

That’s the promise of object-oriented programming: By reusing existing code, you can write less. Moreover, by agreeing to some general conventions, the code that you do write becomes easier to write and easier to read — and thus, easier to maintain.

Sounds great, right? It is, but (of course) there’s a catch: Object-oriented programming has a whole bunch of vocabulary, conventions, and expectations that tend to overwhelm many experienced developers with a background in objects.

And even if you have experience with objects, then Python’s way of doing things might strike you as as bit odd.

In either case, I have a solution for you: Weekly Python Exercise.

If you feel stuck with Python objects, then Weekly Python Exercise A3 (objects for beginners) is for you. We’ll cover such topics as objects, classes, instances, methods, attributes, and inheritance — not with dry lectures, but by actually solving new problems each week. Here’s how it works:

  • Every Tuesday, you’ll receive a problem description, along with some sources to read and “pytests” tests
  • On the following Monday, you’ll receive a detailed solution and explaination
  • In between, you’ll be able to participate in our exclusive forum
  • About once a month, you can join live office hours, to ask me questions and/or review the answers.

After fifteen weeks of working with objects, you’ll know how to write them, but will also understand the ideas behind them. You won’t be stuck any more, checking Stack Overflow a dozen times each day to double-check the syntax for working with objects in Python. Moreover, you’ll see the Pythonic way of doing things, helping you to write code in a way that Python developers aim to achieve.

Hundreds of developers from around the world have already enjoyed Weekly Python Exercise since it started several years ago. WPE A3 (objects for beginners) starts on September 17th, but early-bird pricing for that cohort ends tomorrow, Tuesday, September 3rd.

Questions or comments? Or think that you’re eligible for one of the many discount coupons? Read more at https://WeeklyPythonExercise.com/, or just e-mail me at reuven@lerner.co.il.

2

Proud to be sponsoring PyCon 2020


I’m delighted to announce that Weekly Python Exercise is a gold sponsor of PyCon 2020, to be held in Pittsburgh, Pennsylvania. PyCon is the largest Python conference in the world, and is both fun and interesting for Python developers of all experience levels and backgrounds.

This will be the second year in a row sponsoring PyCon, and my third year attending the conference. Sponsoring means that I’ll not only be there, but that I’ll have a booth — giving away T-shirts and advertising the courses I teach at companies around the world, my online course offerings, and (of course) Weekly Python Exercise.

So if you’re a Python developer, you should attend PyCon. I promise that it’s worth attending.

And if you’re a developer who wants to become more fluent in Python, then check out Weekly Python Exercise. A new beginner-level cohort, focusing on objects, will start on September 17th. And a new advanced-level one, on a grab-bag of topics, will start in late October. Questions or comments? Just e-mail me, at reuven@lerner.co.il.

See you in Pittsburgh!

2

Weekly Python Exercise A3 (beginner objects) is open

Image

If you’ve been programming in Python for any length of time, then you’ve undoubtedly heard that “everything is an object.”

But what does that mean? And who cares?  And what effect does that have on you as a developer — or on Python, as a language?

Indeed, how can (and should) you take advantage of Python’s object-oriented facilities to make your code more readable, maintainable, standard, and (dare I say it) Pythonic?

If you’re relatively new to Python, and have been struggling with some of these same questions, or if you’re just wondering about the differences between instances, classes, methods, and attributes, then I have good news for you: The upcoming cohort of Weekly Python Exercise is all about object-oriented programming.

In this 15-week course, you’ll learn in the best way I know, by solving problems and discussing them with others. As you work through the exercises, you’ll get a better understanding of:

  • Instances and classes
  • Attributes, at both the attribute and class level
  • Methods
  • Composition of objects
  • Inheritance
  • Magic methods

Weekly Python Exercise, of course, is a family of 15-week classes designed to help improve your Python fluency.  Each course works the same:

  • On Tuesday, you receive a problem (along with “pytest” tests)
  • On the following Monday, you receive my detailed solution. 
  • In between, we can discuss it in the forum.
  • I also have live office hours, when people can ask any questions that they might have.

WPE A-level courses are for beginners, while B-level courses are for more advanced Python developers. But you can take any or all of them, in any order — and there’s no overlap between the exercises in these classes and any of the previous books/courses I’ve given.

This new cohort (A3) will be starting on Tuesday, September 17th.  To join, you must sign up before September 10th.  But if you sign up by September 3rd, you’ll get the early-bird discount, bringing the price down to $80 — more than $20 off the full price.

I won’t be offering these exercises for at least one more year. So if you want to sharpen your OO skills before the autumn of 2020, then you should act now.

As always, you can get an even better price if you’re a student, pensioner/retiree/senior citizen, or living permanently outside of the world’s 30 richest countries. Just reply to this e-mail, and I”ll send you the appropriate coupon code.

And if several people (at least five) from your company want to join together?  Let me know, and I’ll give you an additional discount, too.

There’s lots more to say about Weekly Python Exercise, now in its third year of helping Python developers from around the world to write better code — doing more in less time, and getting better jobs than before.  You can read more, and try to some sample exercises, at https://WeeklyPythonExercise.com/ .

But if you’ve always wanted to improve your fluency with Python objects, then you can just sign up at https://WeeklyPythonExercise.com/ .

Don’t wait, though! The early-bird discount ends on September 3rd.

2

Enjoyed the movie? Now you can also enjoy the (Jupyter note)book!

About a month ago, I started my “Python standard library video explainer series” on YouTube. My goal is to walk through the Python standard library, one little bit at a time — explaining it to Python developers, and also discovering (for myself) the many gems that exist in there, but which I’ve never had a chance to discover or work with.

The series now has more than 25 videos, with more than 2.5 hours of content. I’m currently still on the “builtins” area of the standard library, but will soon be making my way into non-builtin modules. I have already learned a lot in preparing this series, and expect to learn much more as I march through the standard library, one little bit at a time.

As is always the case when I teach, I use the Jupyter notebook and live-code as I explain things. One viewer/subscriber suggested that I should share these Jupyter notebooks with the public.

And so, as of earlier today, you can get copies of the Jupyter notebooks I used in making my videos from GitHub: https://github.com/reuven/video-explainer-notebooks . I hope that the combination of Jupyter + videos will help people to understand Python better.

Subscribe to my YouTube channel, and you’ll get updates whenever I add to my explainer series!

2

Weekly Python Exercise is a PyCon Africa 2019 bronze sponsor

I’ve attended two Python conferences so far this year: PyCon (in May, in Cleveland, Ohio) and EuroPython (in July, in Basel, Switzerland). Both were fantastic; I was happy to be a sponsor at PyCon in the US, and to give my “practical decorators” talk at both conferences.

While in Basel, I heard about PyCon Africa, a conference for people from all over Africa to come and share their Python knowledge with one another. And while I couldn’t make it (since I’m giving my “Python for non-programmers” course to a company in the US), I was delighted to become a bronze sponsor of the conference, under the Weekly Python Exercise name.

I hope that this year’s PyCon Africa, which starts today, is so over-the-top successful that it’ll happen again next year — and that I’ll be able to join it in person.

Meanwhile, don’t forget that if you want to improve your Python fluency, then Weekly Python Exercise offers a family of 15-week courses, at both beginner and advanced levels, to help you out. And if you live outside of the world’s 30 richest countries, then you’re entitled to a very steep discount on the enrollment fee. A new beginner-level cohort starts in September; find out more at https://WeeklyPythonExercise.com/ !

1 2 3 14
>