Announcing: Weekly Python Exercise A2 — functions and modules for Python beginners

Image

I spend just about every day teaching Python to people at companies around the world.

I’m always amazed to see just how popular Python is, and how many people are using it — and in how many ways they are using it.

But I’m also amazed by how many people are just “getting by” with Python. You know, they’re able to write some basic functions, and read data from files, and even perform some basic manipulations on their data, without too much help.

But those people are turning to Stack Overflow for just about anything non-trivial. And that might seem fine, except:

  • They’re spending lots of time just searching for the right answer to their questions
  • Then they’re spending lots of time modifying the answer they found online, usually through trial and error
  • Then they’re not really sure what they’ve done, so if it breaks, they’re out of luck.

Does this describe you?  Because it describe a huge number of the people I teach.

These people can use Python, in the same way that you can use a phrasebook to get around in a foreign country whose language you don’t speak. Yes, you can get through some basic tasks. But you’ll never be able to take on big jobs, and you’ll always feel frustrated, or stuck, that you don’t really know what you’re doing.

And maybe you’re even a bit nervous that your boss will discover just how little Python you know.

And besides, let’s face it: There are many problems you can’t Google your way out of.

Fortunately, there is a solution to this problem: Practice.  If you have already learned Python’s basics, but you haven’t learned how to actually use the language, then you need practice.  Just as if you want to learn a language, you need to surround yourself with it, so that you start to think in that language.

Weekly Python Exercise, now in its third year, is my best solution to this problem of Python non-fluency.  Each WPE cohort has 15 exercises (and detailed solutions), which you solve along with others taking the course at the same time as you.  WPE is designed to force you to think in new ways, to become more familiar with Python’s syntax, libraries, and capabilities — and then to be better at your current job, or even (I hope) get a better job in the future.

On May 14th, I’ll be starting a new cohort of Weekly Python Exercise A2.  “A” is the level (for beginners), and this is the 2nd course in the A series.  A2 focuses on Python functions and modules. So we’ll talk about function parameters and defaults, a bit of passing functions as arguments to other functions, and then how to best use the modules in Python’s standard libraries to accomplish your goals.

Registration is only open until May 10th.  So if you want to join this cohort, you should act now!

Click here, and learn more about Weekly Python Exercise!

Improve your Python skills with my new book: Python Workout

A few years ago, I noticed that many of the participants in my corporate Python courses were asking the same question: How can I get additional practice?

And I have to say, this question made a lot of sense. After all, you can only absorb so much information from a course, regardless of how good it is. It’s only through repeated practice that you really gain the mastery and fluency that you need. This is true in sports. This is true in language. This is true in crossword puzzles. And it’s true in programming — even in a language as straightforward as Python.

Thus was born “Practice Makes Python,” my first ebook. That ebook became a course with accompanying videos. Those led me to write another book, a bunch of additional video courses (with many more on the way), and (of course) Weekly Python Exercise, now a family of six 15-week courses.

Well, I have exciting news to announce today: “Practice Makes Python” has undergone massive editing and expansion, and is being republished by Manning as “Python Workout.”

How has it changed?

  • It now uses Python 3 exclusively.
  • I’ve added many diagrams and figures.
  • Just about every exercise has a link to PythonTutor.com, where you can follow the code yourself, line by line.
  • There are numerous sidebars, describing aspects of Python’s functionality that you might not have understood previously.
  • After presenting my solution to an exercise, I then present three additional “beyond the exercise” challenges.
  • It has gone through a lot of editing by people with a great of experience in the editing and publishing worlds.

The book was just released as a MEAP (“Manning Early Access Program”), which means that it’s available as an online book today, with three of the 10 chapters already online. The next three chapters should be released within the next 1-2 months, and the full book should be done (if all goes well) by September or October. The videos are still, for the time being, the old ones that use Python 2 — but will be replaced in the coming months, as well.

If you buy the MEAP, you’ll have access to these updates as they happen, and will also be able to tell me what worked well… and what didn’t. You can be sure that I’m always experimenting with my exercises, trying to figure out how to get the questions, the tasks, and the explanations to be a bit more effective and useful to people.

If this sounds good, then I want to make it even better: As a reader of my blog, you can get 50% off “Python Workout” by using the promo code mllerner50 . Note that this promo code is good for all Manning books, in all formats (online and print). So if you see other things you like, go wild!

Once again: Get “Python Workout” for 50% off with the promo code mllerner50 !

1

Announcing: My new NumPy course is live!

Guess what?  Python is the #1 language for data science.  I know, it doesn’t seem like this should be true.  Python is a great language, and easy to learn, but it’s not the most efficient language, either in execution speed or in its memory usage.

That’s where NumPy comes in: NumPy lets you have the best of both worlds, enjoying Python’s friendliness with C’s efficiency. As a result:

  • Companies are switching from C, C++, and Java to Python — because NumPy allows them to do so, with no loss of execution speed and with huge gains in their programmer productivity.
  • Companies are switching from Matlab to Python — because Python’s open-source license saves them huge amounts of money, and NumPy provides the functionality they need
  • Developers who never saw themselves as analysts or data scientists are learning these disciplines, because NumPy gives them an easy onramp into doing so
  • Students are discovering that you don’t need to choose between a high-level language and ruthless program efficiency, thanks to NumPy.

So, what’s the problem?  Well, NumPy works differently from regular Python data structures.  Learning the ins and outs, and how to apply these ideas to your own work, can take some time, even (or especially) if you have lots of Python experience.

It shouldn’t come as a surprise, then, that my “Intro to data science with Python” course has become one of my most popular.  Companies around the world, from Apple to Ericsson, IBM to PayPal, VMWare to Western Digital, have asked me to teach it to their engineers.  What do I teach on the first day of that course?  NumPy.  Because without NumPy, you can’t do any serious data science with Python. 

Companies keep inviting me back, time after time, to teach this course.  Almost immediately, their people use the techniques I teach to do more in less time — which is, after all, the promise that Python has offered to us.

I’m thus delighted to announce that my new “NumPy” course is available online.  This course includes nearly 5 hours of videos and nearly 60 exercises, designed to help you understand how to use NumPy — along with its companion software, Jupyter and Matplotlib.  It includes the same content as I present to these Fortune 500 companies, but for your own personal use, whenever and wherever you want to learn.

  • If you’re a programmer itching to learn data science, then this course is for you — providing an introduction to data science.
  • If you’re a data scientist interested in learning Python, then this course is for you — showing you how Python can serve your analysis needs.
  • If you’re an analyst who wants to use Python instead of Excel, then this course is for you — giving you a step-by-step introduction to the NumPy library.
  • If your job involves heavy doses of math, then this course is for you — showing you how NumPy can, together with Python, help you write easy-to-maintain code that executes at blazing speeds.

In short: If you want to learn one of the hottest topics in the computer industry, gaining skills that are highly in demand, then this course is for you.

Buy NumPy now

Want to learn more?  Just go to the course page, and see what topics I cover.  You can even watch a few of the videos for free.  And then start your data-science journey with the tool that is getting lots of people excited: NumPy.

Learn more about my NumPy course at https://store.lerner.co.il/numpy .

2

Python’s “else” clause for loops

Let’s say that we have a list of tuples, with each tuple containing some numbers. For example:

>>> mylist = [(3,5), (2,4,6,8), 
(4,10, 17), (15, 14, 11), (3,3,2)]

I want to write a program that asks the user to enter a number. If one of the tuples adds up to the user’s input, we’ll print the tuple out. Sounds good, right? Here’s how we could write it:

>>> lookfor = int(input("Enter a number: "))
for one_tuple in mylist:
if sum(one_tuple) == lookfor:
print(f"{one_tuple} adds up to {lookfor}!")

In other words: We iterate over the list of tuples, and use the built-in “sum” function to sum its contents. When we find a tuple whose sum matches the user’s input, “lookfor”, we print the tuple.

And sure enough, if we run this short program with our previously defined “mylist”, we get the following:

Enter a number: 8
(3, 5) adds up to 8!
(3, 3, 2) adds up to 8!

But what if the user enters a number, and no tuples match that sum? We don’t say anything; we basically just leave the user hanging. It would be nice to let them know that none of the tuples in our list matched their sum, right?

This can traditionally be done using a “flag” variable, one whose value starts off with a “False” value (i.e., we haven’t found what we’re searching for) and which is set to “True” if and when we do find what we want. Here’s how we could do that in Python:

found = False
lookfor = int(input("Enter a number: "))
for one_tuple in mylist:
if sum(one_tuple) == lookfor:
found = True
print(f"{one_tuple} adds up to {lookfor}!")
if not found:
print(f"Sorry, didn't find any tuple summing to {lookfor}")

And sure enough, this works just fine:

Enter a number: 3
Sorry, didn't find any tuple summing to 3

Python provides us with a different way to handle this situation. It’s a bit odd at first, and newcomers find it hard to understand — but in cases like this, it helps to shorten the code.

What I’m talking about is the “else” clause for loops. You’re undoubtedly familiar with the “else” clause for “if” statements. You know, the block that gets executed if the main “if” clause isn’t true.

With a loop — and this works with both “for” and “while” loops — the “else” clause is executed if the loop reached its natural end. That is, if Python didn’t encounter a “break” statement in your loop, then the “else” clause will work. This allows us to remove the “found” flag variable, as follows:

lookfor = int(input("Enter a number: "))
for one_tuple in mylist:
if sum(one_tuple) == lookfor:
print(f"{one_tuple} adds up to {lookfor}!")
break
else:
print(f"Sorry, didn't find any tuple summing to {lookfor}")

In other words: If we found a tuple that adds up to “lookfor”, then we print it out and exit the loop immediately, using “break”. But if we reach the natural end of the loop (i.e., never encounter a “break”), then the “else” clause is executed.

Aha, but you might see a problem with our implementation: Whereas in the first and second versions, we printed all of the matching tuples, in this version, we only print the first matching tuple. Even though two tuples have sums of 8, our “break” statement exits the loop once it finds the first one.

The “else” clause on a loop is thus only useful if you’re planning to exit out of it with a “break” statement at some point. If there’s no “break”, then there’s no real value in having an “else” statement.

Also realize that there’s a difference between the code in an “else” block and the code that immediately follows a loop. Code in the “else” block will only execute if the “break” wasn’t encountered.

I’ve found that many newcomers to Python are confused about these “else” blocks, partly because they expect “else” to be connected to an “if”. They thus try to align the “else” with the “if” statement’s indentation, which doesn’t work. In many ways, I do wish that Python’s core developers had chosen a different word, rather than reusing “else”… but that didn’t happen, and so now we have to learn that “else” can be used in two different contexts.

1

Seven ways to improve your team’s Python

If you’re a manager, then you’re always trying to find ways that’ll help your team do more in less time. That’s why you use Python — because it makes your developers more productive. They can spend more time creating new features, and less time debugging or maintaining existing code. It’s no surprise that so many companies are moving to Python.

After you’ve moved to Python, you can still make your team more effective. That is, your organization can become more productive, combining technology and culture to help your developers improve. In such a scenario, everyone wins: Your company becomes more efficient and effective, and your team members are more satisfied.

Get the bonus content: Seven ways to improve your team’s Python


How can you improve your company’s Python? I’ve prepared a 6-page white paper on the subject, describing a variety of techniques I’ve both seen and used with companies around the world. It’s free to download; I hope that you’ll read it and adopt some of the techniques I’ve listed here.

Thoughts or feedback on what I’ve written? You can always e-mail me at reuven@lerner.co.il., or contact me on Twitter as @reuvenmlerner. I’ll be happy to hear your thoughts!

Registration for Weekly Python Exercise ends in 24 hours

Weekly Python Exercise logo

Registration for Weekly Python Exercise B1 (i.e., advanced level, part 1) closes in about 24 hours. Don’t be left out!

WPE gives you exercises in all sorts of advanced Python topics: Data structures, functions, object-oriented programming, comprehensions, generators, and decorators.    These exercises model real-world problems, so that when you encounter problems at your job, you’ll be able to attack them better and faster, with fewer searches on Stack Overflow and Google.

Moreover, it does so with an online community — so you can compare code with, and learn from, others in the cohort.

You will, after 15 weeks in WPE, be a better, more fluent Python developer. You’ll be more valuable at your current job, and able to get better jobs in the future.  But don’t just take my word for it.  Here’s what some WPE students have said:

  • Currently I’m engaged mid way through the A1 cohort and what differentiates Reuven’s course from regular online reading, YouTube videos or other self study mechanisms are context and application.
  • Fully recommend the course to anyone wanting to not only begin with Python, but learn it contextually and apply the learning via best practices.
  • I’m 20-something weeks into my first WPE course and really enjoy it. It’s totally worth it and a fantastic way to keep my Python skills sharp and learn new things in the process!

Hundreds of Python programmers from around the world have already leveled up their Python knowledge with Weekly Python Exercise.  You can, too!

To learn more, go to http://WeeklyPythonExercise.com/

Or join me at a free Webinar later today, at which I’ll answer any questions you might have. To see when the Webinar is happening in your time zone, click here:

https://www.timeanddate.com/worldclock/fixedtime.html?msg=WPE+Webinar&iso=20190307T22&p1=110&ah=1

To join the Webinar itself, use the following Zoom URL:

https://zoom.us/j/220787502

Any questions or comments? Just contact me, at reuven@lerner.co.il or on Twitter as @reuvenmlerner.

Understanding Python slices

Let’s say that you have a Python string, and want to grab a substring from it. The best way to do so is with a “slice”:

>>> s = 'abcdefghij'
>>> print(s[3:8])
defgh

In the above code, we’ve defined a string. We’ve then asked for the string to be returned, starting at index 3 and up to (and not including) index 8. Sure enough, we get the string ‘defgh’ back.

Slices work on all sequences in Python. Which means that you can have a slice of a string, list, or tuple, and you’ll have the same effect:

>>> mylist = [10, 20, 30, 40, 50, 60, 70, 80, 90, 100]
>>> mylist[3:7]
[40, 50, 60, 70]

While “for” loops in Python don’t explicitly use numeric indexes, slices do. My students often ask me how, without indexes, they can iterate over only part of a string (or other sequence). The answer is to use a slice. For example, if I’m interested in iterating over letters 3-7 in my string “s” from above, I won’t (and can’t) use a C-like “for” loop, starting the loop at index 3 and ending at index 7. Rather, I need to run my “for” loop over a complete string… which a slice just happens to return to me:

for one_letter in s[3:7]:   # s[3:7] returns a new string
print(one_letter)

Remember that a slice from type X will always return a new object of type X. Thus, slicing a string returns a string, while slicing a list returns a list and slicing a tuple returns a tuple. For example:

>>> mylist[3]           # one index
40
>>> mylist[3:7] # slice
[40, 50, 60, 70]
>>> mylist[3] * 2 # Multiply the element at mylist[3] by 2
80
>>> mylist[3:7] * 2 Multiply the slice (list) at mylist[3:7] by 2
[40, 50, 60, 70, 40, 50, 60, 70]

Slice syntax is more flexible than this: You can leave off the starting index, ending index, or both to indicate that you want to go all the way to the extreme edge:

>>> s
'abcdefghij'
>>> s[:5] # from the start until (and not including) index 5
'abcde'
>>> s[5:] # from index 5 until (and through) the end
'fghij'
>>> s[5:-1] # not the same as s[5:] -- doesn't include the end
'fghi'

Because slices create new objects, you can sometimes use them to avoid problems with mutable data:

>>> mylist = [10, 20, 30]
>>> biglist = [mylist, mylist]
>>> mylist[0] = '!'
>>> biglist
[['!', 20, 30], ['!', 20, 30]]

In the above code, changing “mylist” affected “biglist”, because both names were still pointing (one directly, one indirectly) to the original list. By contrast, if I use slices, I can avoid this:

>>> mylist = [10, 20, 30]
>>> biglist = [mylist[:], mylist[:]] # put copies, not mylist, in biglist
>>> mylist[0] = '!'
>>> biglist
[[10, 20, 30], [10, 20, 30]

Note that this isn’t a perfect solution; if you’re copying complex data structures, then you’ll probably want to look at the “copy” module, and explore its “copy” and “deepcopy” methods.

By default, a slice uses each element from the sequence on which it’s working. But what if we only want every other element? Then we can use the (optional) third part of slice syntax, known as either the “step size” or the “stride.” By default, the step size is 1. But by adding another number after a second colon, we can modify this, too:

>>> mylist = [10, 20, 30, 40, 50, 60, 70, 80]
>>> mylist[2:7] # step size of 1 (default)
[30, 40, 50, 60, 70]
mylist[2:7:2] # step size of 2
[30, 50, 70]

With the combination of start-stop-stride, we can work with all sorts of combinations of things:

>>> import string
>>> s = string.ascii_lowercase
>>> s[5:20:3] # from 5, until (not including) 20, step size 3
'filor'
>>> s[2:25:4] # from 2, until (not including) 25, step size 4
'cgkosw'
>>> s[:25:4] # from start, until (not including) 25, step size 4
'aeimquy'
>>> s[2::4] # from 2, through the end, step size 4
'cgkosw'
>>> s[2::2] # from 2, through the end, step size 2
'cegikmoqsuwy

It gets even better when you discover that the step size can be negative, which allows us to retrieve values in reverse order from the original data structure. Just remember, in such cases, that the start needs to be bigger than the end:

>>> s[20:5:-1]        # from 20 to 5 (not including), step size 1
'utsrqponmlkjihg'
>>> s[20:5:-4] # from 20 to 5 (not including), step size 4
'uqmi'
>>> s[:5:-4] # from the end to 5 (not including), step size 4
'zvrnj'
>>> s[20::-2] # from 20 to the beginning, step size 2
'usqomkigeca'
>>> s[::-1] # no indexes? from end to start, backwards
'zyxwvutsrqponmlkjihgfedcba'
>>> s[5:20:-5] # Start is smaller, and negative step? Bad news...
''

Just as you can use variables in an index, you can also use them in a slice:

>>> s = 'abcdefghij'
>>> i = 3
>>> j = 6
>>> s[i]
'd'
>>> s[i:j]
'def'
>>> s[::i]
'adgj'

Normally, if you try to retrieve from an index that’s beyond the bounds of the data structure, you’ll get an error:

>>> s = 'abcdefghij'
>>> s[15]
IndexError: string index out of range

But slices are far more forgiving; if you go off of the edge of a slice, then Python will simply stop at the edge of your sequence:

>>> s = 'abcdefghij'
>>> s[:15]
'abcdefghij'

You can also assign to slices. For the most part, this means modifying multiple elements in a list:

>>> mylist = [10, 20, 30, 40, 50, 60, 70]
>>> mylist[3:5]
[40, 50]
>>> mylist[3:5] = 'XY'
>>> mylist
[10, 20, 30, 'X', 'Y', 60, 70]

Notice, in the above example, that we assigned to a slice from a string. So long as the item on the right side is iterable, you can assign it to a slice.

You can expand and contract a list by assigning more or fewer item to a slice:

>>> mylist = [10, 20, 30, 40, 50, 60, 70]
>>> mylist[3:5] = 'WXYZ'
>>> mylist
[10, 20, 30, 'W', 'X', 'Y', 'Z', 60, 70]

In the above example, our slice was only two items long. However, we assigned four elements to it. This meant that the list grew as a result of our assignment. We can similarly shrink it:

>>> mylist
[10, 20, 30, 'W', 'X', 'Y', 'Z', 60, 70]
>>> mylist[3:7] = [99, 98]
>>> mylist
[10, 20, 30, 99, 98, 60, 70]

While we normally think about only sequences as being sliceable, other objects can potentially work with slices, too. For example, the “range” object in Python 3 is sliceable:

>>> r = range(100, 300, 3)
>>> r[4:8]
range(112, 124, 3)
>>> r[10:20:2]
range(130, 160, 6)

Notice that the printed representation of a range object includes the three parts that we’ve discussed in a slice: The start, the end (+1), and the step size.

And indeed, Python provides a “slice” builtin that I’ve never used, but which I can imagine would be useful if you want to reuse a slice multiple times:

>>> s = 'abcdefghij'
>>> myslice = slice(2, 8, 2)
>>> s[myslice]
'ceg'
>>> s[2:8:2]
'ceg'

>>> myslice = slice(None, None, -2)
>>> s[myslice]
'jhfdb'
>>> s[::-2]
'jhfdb'

Notice how our “slice” object works just like the start:end:step syntax; if you want to indicate the edge (via nothing between the colons), then you can use “None”. Again, I’m not really sure why you would need a slice object, but it’s nice to know that everything in Python is indeed an object, and that the :: syntax is translated into a slice object in the end.

What if you want your own objects to be sliceable? Truth be told, there’s not much to do: The __getitem__ method is used for retrieving individual items as well as slices; while there used to be a __getslice__ method, nowadays you are expected to write __getitem__ such that it handles individual indexes and slices. In many cases, that’s trivially easy to do:

class Foo():
def init(self, x):
self.x = x
def getitem(self, index):
return self.x[index]

>>> f = Foo('abcdefghij')
>>> f[3]
'd'
>> f[3:5]
'de'

Of course, if you want to do something more sophisticated than returning one or many elements from your object, then you’ll have to work a bit harder. But with an if/else, you can make such decisions, and then return the appropriate data.

Slices are both common and convenient ways to extract portions of Python data structures — usually with builtin objects, but also on your own. Once you get used to slices, you’ll see lots of uses for them, and wonder how you got along without them.

Want to improve your Python skills? Join the upcoming cohort of Weekly Python Exercise!

For more than 20 years, I’ve been teaching Python courses to companies around the world. This means that just about every day, I’m on the front lines of Python learning.  I see, first-hand, what companies want people to learn and also what people are struggling to understand.

Weekly Python Exercise logo

The result is Weekly Python Exercise, my course that’s designed to make you more fluent in Python by giving you (surprise, surprise) a weekly Python exercise.

I’m starting a new advanced (B) level cohort on Tuesday, March 12th.   Over 15 weeks, participants in the cohort will improve their understanding of Python data structures, objects, threads, functions, iterators, and more.  Plus, every exercise now comes with automated tests written with “pytest” — so you can not only check if your code fits the specs I’ve provided, but also learn more about how to use pytest!

If you’ve always wanted to improve your Python, then there’s no better way to do it than WPE.  Want to learn more?  Just go to WeeklyPythonExercise.com. From that page, you can learn about WPE, sign up for a free sample version (with two exercises), and even register for the course.

If you’ve felt stuck with Python and have always wanted to push your Python skills ahead, then I encourage you to learn more about the B1 (advanced level, part 1) cohort that will start in March.

Learn more about Weekly Python Exercise

The “Train Better” podcast is live!

I’m a professional trainer; just about every day, I’m in a different city, country, and/or company teaching Python, data science, and other topics. Over the last few years, I’ve also thought a lot about training as a specialty — and I write about it on my “Trainer Weekly” newsletter, as well as in a “Technical training” Facebook group.

I’ve now launched my “Train Better” podcast, which aims to give advice about the business, pedagogy, and logistics of the training industry. I’ll also interview trainers at various stages of their career, so that we can learn from their successes and failures.

Click here: Train Better podcast

If you’re interested in training, or are already training and want to get better at it, then I hope you’ll enjoy the podcast. If you have suggestions for topics or guests, then please contact me, as well!

Answering Python questions from readers

Every so often, I’ve asked readers of my free, weekly “Better developers” newsletter to send me their Python problems. And every so often, I get a chance to answer their questions, going through their Python problems and trying to solve them.

I’ve recently recorded and uploaded two videos with solutions to their problems, which I’m sharing here.

Have Python problems you want to solve? Send them to me at reuven@lerner.co.il — and if I choose your question, I’ll give you 30% off any course in my online catalog.

Question 1: Finding links in an e-mail newsletter, and saving them to an Excel spreadsheet

Question 2: Simple PostgreSQL queries from Python


1 3 4 5 6 7 18
>