March 24, 2019 , bY reuven
Python

Let’s say that we have a list of tuples, with each tuple containing some numbers. For example:

>>> mylist = [(3,5), (2,4,6,8), 
(4,10, 17), (15, 14, 11), (3,3,2)]

I want to write a program that asks the user to enter a number. If one of the tuples adds up to the user’s input, we’ll print the tuple out. Sounds good, right? Here’s how we could write it:

>>> lookfor = int(input("Enter a number: "))
for one_tuple in mylist:
if sum(one_tuple) == lookfor:
print(f"{one_tuple} adds up to {lookfor}!")

In other words: We iterate over the list of tuples, and use the built-in “sum” function to sum its contents. When we find a tuple whose sum matches the user’s input, “lookfor”, we print the tuple.

And sure enough, if we run this short program with our previously defined “mylist”, we get the following:

Enter a number: 8
(3, 5) adds up to 8!
(3, 3, 2) adds up to 8!

But what if the user enters a number, and no tuples match that sum? We don’t say anything; we basically just leave the user hanging. It would be nice to let them know that none of the tuples in our list matched their sum, right?

This can traditionally be done using a “flag” variable, one whose value starts off with a “False” value (i.e., we haven’t found what we’re searching for) and which is set to “True” if and when we do find what we want. Here’s how we could do that in Python:

found = False
lookfor = int(input("Enter a number: "))
for one_tuple in mylist:
if sum(one_tuple) == lookfor:
found = True
print(f"{one_tuple} adds up to {lookfor}!")
if not found:
print(f"Sorry, didn't find any tuple summing to {lookfor}")

And sure enough, this works just fine:

Enter a number: 3
Sorry, didn't find any tuple summing to 3

Python provides us with a different way to handle this situation. It’s a bit odd at first, and newcomers find it hard to understand — but in cases like this, it helps to shorten the code.

What I’m talking about is the “else” clause for loops. You’re undoubtedly familiar with the “else” clause for “if” statements. You know, the block that gets executed if the main “if” clause isn’t true.

With a loop — and this works with both “for” and “while” loops — the “else” clause is executed if the loop reached its natural end. That is, if Python didn’t encounter a “break” statement in your loop, then the “else” clause will work. This allows us to remove the “found” flag variable, as follows:

lookfor = int(input("Enter a number: "))
for one_tuple in mylist:
if sum(one_tuple) == lookfor:
print(f"{one_tuple} adds up to {lookfor}!")
break
else:
print(f"Sorry, didn't find any tuple summing to {lookfor}")

In other words: If we found a tuple that adds up to “lookfor”, then we print it out and exit the loop immediately, using “break”. But if we reach the natural end of the loop (i.e., never encounter a “break”), then the “else” clause is executed.

Aha, but you might see a problem with our implementation: Whereas in the first and second versions, we printed all of the matching tuples, in this version, we only print the first matching tuple. Even though two tuples have sums of 8, our “break” statement exits the loop once it finds the first one.

The “else” clause on a loop is thus only useful if you’re planning to exit out of it with a “break” statement at some point. If there’s no “break”, then there’s no real value in having an “else” statement.

Also realize that there’s a difference between the code in an “else” block and the code that immediately follows a loop. Code in the “else” block will only execute if the “break” wasn’t encountered.

I’ve found that many newcomers to Python are confused about these “else” blocks, partly because they expect “else” to be connected to an “if”. They thus try to align the “else” with the “if” statement’s indentation, which doesn’t work. In many ways, I do wish that Python’s core developers had chosen a different word, rather than reusing “else”… but that didn’t happen, and so now we have to learn that “else” can be used in two different contexts.

March 22, 2019 , bY reuven
Python

If you’re a manager, then you’re always trying to find ways that’ll help your team do more in less time. That’s why you use Python — because it makes your developers more productive. They can spend more time creating new features, and less time debugging or maintaining existing code. It’s no surprise that so many companies are moving to Python.

After you’ve moved to Python, you can still make your team more effective. That is, your organization can become more productive, combining technology and culture to help your developers improve. In such a scenario, everyone wins: Your company becomes more efficient and effective, and your team members are more satisfied.

Get the bonus content: Seven ways to improve your team’s Python


How can you improve your company’s Python? I’ve prepared a 6-page white paper on the subject, describing a variety of techniques I’ve both seen and used with companies around the world. It’s free to download; I hope that you’ll read it and adopt some of the techniques I’ve listed here.

Thoughts or feedback on what I’ve written? You can always e-mail me at reuven@lerner.co.il., or contact me on Twitter as @reuvenmlerner. I’ll be happy to hear your thoughts!

March 7, 2019 , bY reuven
Python

Weekly Python Exercise logo

Registration for Weekly Python Exercise B1 (i.e., advanced level, part 1) closes in about 24 hours. Don’t be left out!

WPE gives you exercises in all sorts of advanced Python topics: Data structures, functions, object-oriented programming, comprehensions, generators, and decorators.    These exercises model real-world problems, so that when you encounter problems at your job, you’ll be able to attack them better and faster, with fewer searches on Stack Overflow and Google.

Moreover, it does so with an online community — so you can compare code with, and learn from, others in the cohort.

You will, after 15 weeks in WPE, be a better, more fluent Python developer. You’ll be more valuable at your current job, and able to get better jobs in the future.  But don’t just take my word for it.  Here’s what some WPE students have said:

  • Currently I’m engaged mid way through the A1 cohort and what differentiates Reuven’s course from regular online reading, YouTube videos or other self study mechanisms are context and application.
  • Fully recommend the course to anyone wanting to not only begin with Python, but learn it contextually and apply the learning via best practices.
  • I’m 20-something weeks into my first WPE course and really enjoy it. It’s totally worth it and a fantastic way to keep my Python skills sharp and learn new things in the process!

Hundreds of Python programmers from around the world have already leveled up their Python knowledge with Weekly Python Exercise.  You can, too!

To learn more, go to http://WeeklyPythonExercise.com/

Or join me at a free Webinar later today, at which I’ll answer any questions you might have. To see when the Webinar is happening in your time zone, click here:

https://www.timeanddate.com/worldclock/fixedtime.html?msg=WPE+Webinar&iso=20190307T22&p1=110&ah=1

To join the Webinar itself, use the following Zoom URL:

https://zoom.us/j/220787502

Any questions or comments? Just contact me, at reuven@lerner.co.il or on Twitter as @reuvenmlerner.

March 4, 2019 , bY reuven
Python

Let’s say that you have a Python string, and want to grab a substring from it. The best way to do so is with a “slice”:

>>> s = 'abcdefghij'
>>> print(s[3:8])
defgh

In the above code, we’ve defined a string. We’ve then asked for the string to be returned, starting at index 3 and up to (and not including) index 8. Sure enough, we get the string ‘defgh’ back.

Slices work on all sequences in Python. Which means that you can have a slice of a string, list, or tuple, and you’ll have the same effect:

>>> mylist = [10, 20, 30, 40, 50, 60, 70, 80, 90, 100]
>>> mylist[3:7]
[40, 50, 60, 70]

While “for” loops in Python don’t explicitly use numeric indexes, slices do. My students often ask me how, without indexes, they can iterate over only part of a string (or other sequence). The answer is to use a slice. For example, if I’m interested in iterating over letters 3-7 in my string “s” from above, I won’t (and can’t) use a C-like “for” loop, starting the loop at index 3 and ending at index 7. Rather, I need to run my “for” loop over a complete string… which a slice just happens to return to me:

for one_letter in s[3:7]:   # s[3:7] returns a new string
print(one_letter)

Remember that a slice from type X will always return a new object of type X. Thus, slicing a string returns a string, while slicing a list returns a list and slicing a tuple returns a tuple. For example:

>>> mylist[3]           # one index
40
>>> mylist[3:7] # slice
[40, 50, 60, 70]
>>> mylist[3] * 2 # Multiply the element at mylist[3] by 2
80
>>> mylist[3:7] * 2 Multiply the slice (list) at mylist[3:7] by 2
[40, 50, 60, 70, 40, 50, 60, 70]

Slice syntax is more flexible than this: You can leave off the starting index, ending index, or both to indicate that you want to go all the way to the extreme edge:

>>> s
'abcdefghij'
>>> s[:5] # from the start until (and not including) index 5
'abcde'
>>> s[5:] # from index 5 until (and through) the end
'fghij'
>>> s[5:-1] # not the same as s[5:] -- doesn't include the end
'fghi'

Because slices create new objects, you can sometimes use them to avoid problems with mutable data:

>>> mylist = [10, 20, 30]
>>> biglist = [mylist, mylist]
>>> mylist[0] = '!'
>>> biglist
[['!', 20, 30], ['!', 20, 30]]

In the above code, changing “mylist” affected “biglist”, because both names were still pointing (one directly, one indirectly) to the original list. By contrast, if I use slices, I can avoid this:

>>> mylist = [10, 20, 30]
>>> biglist = [mylist[:], mylist[:]] # put copies, not mylist, in biglist
>>> mylist[0] = '!'
>>> biglist
[[10, 20, 30], [10, 20, 30]

Note that this isn’t a perfect solution; if you’re copying complex data structures, then you’ll probably want to look at the “copy” module, and explore its “copy” and “deepcopy” methods.

By default, a slice uses each element from the sequence on which it’s working. But what if we only want every other element? Then we can use the (optional) third part of slice syntax, known as either the “step size” or the “stride.” By default, the step size is 1. But by adding another number after a second colon, we can modify this, too:

>>> mylist = [10, 20, 30, 40, 50, 60, 70, 80]
>>> mylist[2:7] # step size of 1 (default)
[30, 40, 50, 60, 70]
mylist[2:7:2] # step size of 2
[30, 50, 70]

With the combination of start-stop-stride, we can work with all sorts of combinations of things:

>>> import string
>>> s = string.ascii_lowercase
>>> s[5:20:3] # from 5, until (not including) 20, step size 3
'filor'
>>> s[2:25:4] # from 2, until (not including) 25, step size 4
'cgkosw'
>>> s[:25:4] # from start, until (not including) 25, step size 4
'aeimquy'
>>> s[2::4] # from 2, through the end, step size 4
'cgkosw'
>>> s[2::2] # from 2, through the end, step size 2
'cegikmoqsuwy

It gets even better when you discover that the step size can be negative, which allows us to retrieve values in reverse order from the original data structure. Just remember, in such cases, that the start needs to be bigger than the end:

>>> s[20:5:-1]        # from 20 to 5 (not including), step size 1
'utsrqponmlkjihg'
>>> s[20:5:-4] # from 20 to 5 (not including), step size 4
'uqmi'
>>> s[:5:-4] # from the end to 5 (not including), step size 4
'zvrnj'
>>> s[20::-2] # from 20 to the beginning, step size 2
'usqomkigeca'
>>> s[::-1] # no indexes? from end to start, backwards
'zyxwvutsrqponmlkjihgfedcba'
>>> s[5:20:-5] # Start is smaller, and negative step? Bad news...
''

Just as you can use variables in an index, you can also use them in a slice:

>>> s = 'abcdefghij'
>>> i = 3
>>> j = 6
>>> s[i]
'd'
>>> s[i:j]
'def'
>>> s[::i]
'adgj'

Normally, if you try to retrieve from an index that’s beyond the bounds of the data structure, you’ll get an error:

>>> s = 'abcdefghij'
>>> s[15]
IndexError: string index out of range

But slices are far more forgiving; if you go off of the edge of a slice, then Python will simply stop at the edge of your sequence:

>>> s = 'abcdefghij'
>>> s[:15]
'abcdefghij'

You can also assign to slices. For the most part, this means modifying multiple elements in a list:

>>> mylist = [10, 20, 30, 40, 50, 60, 70]
>>> mylist[3:5]
[40, 50]
>>> mylist[3:5] = 'XY'
>>> mylist
[10, 20, 30, 'X', 'Y', 60, 70]

Notice, in the above example, that we assigned to a slice from a string. So long as the item on the right side is iterable, you can assign it to a slice.

You can expand and contract a list by assigning more or fewer item to a slice:

>>> mylist = [10, 20, 30, 40, 50, 60, 70]
>>> mylist[3:5] = 'WXYZ'
>>> mylist
[10, 20, 30, 'W', 'X', 'Y', 'Z', 60, 70]

In the above example, our slice was only two items long. However, we assigned four elements to it. This meant that the list grew as a result of our assignment. We can similarly shrink it:

>>> mylist
[10, 20, 30, 'W', 'X', 'Y', 'Z', 60, 70]
>>> mylist[3:7] = [99, 98]
>>> mylist
[10, 20, 30, 99, 98, 60, 70]

While we normally think about only sequences as being sliceable, other objects can potentially work with slices, too. For example, the “range” object in Python 3 is sliceable:

>>> r = range(100, 300, 3)
>>> r[4:8]
range(112, 124, 3)
>>> r[10:20:2]
range(130, 160, 6)

Notice that the printed representation of a range object includes the three parts that we’ve discussed in a slice: The start, the end (+1), and the step size.

And indeed, Python provides a “slice” builtin that I’ve never used, but which I can imagine would be useful if you want to reuse a slice multiple times:

>>> s = 'abcdefghij'
>>> myslice = slice(2, 8, 2)
>>> s[myslice]
'ceg'
>>> s[2:8:2]
'ceg'

>>> myslice = slice(None, None, -2)
>>> s[myslice]
'jhfdb'
>>> s[::-2]
'jhfdb'

Notice how our “slice” object works just like the start:end:step syntax; if you want to indicate the edge (via nothing between the colons), then you can use “None”. Again, I’m not really sure why you would need a slice object, but it’s nice to know that everything in Python is indeed an object, and that the :: syntax is translated into a slice object in the end.

What if you want your own objects to be sliceable? Truth be told, there’s not much to do: The __getitem__ method is used for retrieving individual items as well as slices; while there used to be a __getslice__ method, nowadays you are expected to write __getitem__ such that it handles individual indexes and slices. In many cases, that’s trivially easy to do:

class Foo():
def init(self, x):
self.x = x
def getitem(self, index):
return self.x[index]

>>> f = Foo('abcdefghij')
>>> f[3]
'd'
>> f[3:5]
'de'

Of course, if you want to do something more sophisticated than returning one or many elements from your object, then you’ll have to work a bit harder. But with an if/else, you can make such decisions, and then return the appropriate data.

Slices are both common and convenient ways to extract portions of Python data structures — usually with builtin objects, but also on your own. Once you get used to slices, you’ll see lots of uses for them, and wonder how you got along without them.

February 26, 2019 , bY reuven
Python

For more than 20 years, I’ve been teaching Python courses to companies around the world. This means that just about every day, I’m on the front lines of Python learning.  I see, first-hand, what companies want people to learn and also what people are struggling to understand.

Weekly Python Exercise logo

The result is Weekly Python Exercise, my course that’s designed to make you more fluent in Python by giving you (surprise, surprise) a weekly Python exercise.

I’m starting a new advanced (B) level cohort on Tuesday, March 12th.   Over 15 weeks, participants in the cohort will improve their understanding of Python data structures, objects, threads, functions, iterators, and more.  Plus, every exercise now comes with automated tests written with “pytest” — so you can not only check if your code fits the specs I’ve provided, but also learn more about how to use pytest!

If you’ve always wanted to improve your Python, then there’s no better way to do it than WPE.  Want to learn more?  Just go to WeeklyPythonExercise.com. From that page, you can learn about WPE, sign up for a free sample version (with two exercises), and even register for the course.

If you’ve felt stuck with Python and have always wanted to push your Python skills ahead, then I encourage you to learn more about the B1 (advanced level, part 1) cohort that will start in March.

Learn more about Weekly Python Exercise

February 20, 2019 , bY reuven
Training

I’m a professional trainer; just about every day, I’m in a different city, country, and/or company teaching Python, data science, and other topics. Over the last few years, I’ve also thought a lot about training as a specialty — and I write about it on my “Trainer Weekly” newsletter, as well as in a “Technical training” Facebook group.

I’ve now launched my “Train Better” podcast, which aims to give advice about the business, pedagogy, and logistics of the training industry. I’ll also interview trainers at various stages of their career, so that we can learn from their successes and failures.

Click here: Train Better podcast

If you’re interested in training, or are already training and want to get better at it, then I hope you’ll enjoy the podcast. If you have suggestions for topics or guests, then please contact me, as well!

February 18, 2019 , bY reuven
Python

Every so often, I’ve asked readers of my free, weekly “Better developers” newsletter to send me their Python problems. And every so often, I get a chance to answer their questions, going through their Python problems and trying to solve them.

I’ve recently recorded and uploaded two videos with solutions to their problems, which I’m sharing here.

Have Python problems you want to solve? Send them to me at reuven@lerner.co.il — and if I choose your question, I’ll give you 30% off any course in my online catalog.

Question 1: Finding links in an e-mail newsletter, and saving them to an Excel spreadsheet

Question 2: Simple PostgreSQL queries from Python


February 17, 2019 , bY reuven
Python

Let’s say that I want to write some Python code that invites the user to enter a number, and then prints that number, tripled. We could say:

>>> n = input("Enter a number: ")
>>> print(f"{n} * 3 = {n*3}")

The good news is that this code works just fine. The bad news is that it probably doesn’t do what you might expect. If I run this program, I’ll see:

Enter a number: 5
5 * 3 = 555

The reason for this output is that the “input” function always returns a string. So sure, we asked the user for a number, but we got the string ‘5’, rather than the integer 5. The ‘555’ output is thanks to the fact that you can multiply strings in Python by integers, getting a longer string back. So ‘a’ * 5 will give us ‘aaaaa’.

Of course, we can always create an integer from a string by applying the “int” class to the user’s input:

>>> n = input("Enter a number: ")
>>> n = int(n)
>>> print(f"{n} * 3 = {n*3}")

Sure enough, we get the following output:

Enter a number: 5
5 * 3 = 15

Great, right? But what happens if the user gives us something that’s no longer numeric? The program will blow up:

Enter a number: abcd

ValueError: invalid literal for int() with base 10: 'abcd'

Clearly, we want to avoid this problem. You could make a good argument that in this case, it’s probably best to run the conversion inside of a “try” block, and trap any exception that we might get.

But there’s another way to test this, one which I use with my into Python classes, before we’ve covered exceptions: Strings have a great method called “isdigit” that we can run, to tell us whether a string contains only digits (0-9), or if it contains something else. For example:

>>> '1234'.isdigit()
True

>>> '1234 '.isdigit() # space at the end
False

>>> '1234a'.isdigit() # letter at the end
False

>>> 'a1234'.isdigit() # letter at the start
False

>>> '12.34'.isdigit() # decimal point
False

>>> ''.isdigit() # empty string
False

If you know regular expressions, then you can see that str.isdigit returns True for ‘^\d+$’. Which can be very useful, as we can see here:

>>> n = input("Enter a number: ")
>>> if n.isdigit():
n = int(n)
print(f"{n} * 3 = {n*3}")

But wait: Python also includes another method, str.isnumeric. And it’s not at all obvious, at least at first, what the difference is between them, because they would seem to give the same results:

>>> n = input("Enter a number: ")
>>> if n.isnumeric():
n = int(n)
print(f"{n} * 3 = {n*3}")

So, what’s the difference? It’s actually pretty straightforward, but took some time for me to find out: Bascially, str.isdigit only returns True for what I said before, strings containing solely the digits 0-9.

By contrast, str.isnumeric returns True if it contains any numeric characters. When I first read this, I figured that it would mean decimal points and minus signs — but no! It’s just the digits 0-9, plus any character from another language that’s used in place of digits.

For example, we’re used to writing numbers with Arabic numerals. But there are other languages that traditionally use other characters. For example, in Chinese, we count 1, 2, 3, 4, 5 as 一,二,三,四, 五. It turns out that the Chinese characters for numbers will return False for str.isdigit, but True for str.isnumeric, behaving differently from their 0-9 counterparts:

>>> '12345'.isdigit()
True

>>> '12345'.isnumeric()
True

>>> '一二三四五'.isdigit()
False

>>> '一二三四五'.isnumeric()
True

So, which should you use? For most people, “isdigit” is probably a better choice, simply because it’s more clearly what you likely want. Of course, if you want to accept other types of numerals and numeric characters, then “isnumeric” is better. But if you’re interested in turning strings into integers, then you’re probably safer using “isdigit”, just in case someone tries to enter something else:

>>> int('二')
ValueError: invalid literal for int() with base 10: '二'

Just when I thought I was done with this, David Beazley reminded me that there’s a third method I should be dealing with: str.isdecimal. This asks a slightly different question, namely whether we have something which is a number but not a decimal number. What does this mean?

Well, in Python, we can describe “2 to the 2nd power” as “2 ** 2”. But if you want to print it, you’ll probably use something a bit fancier, such as 2². This is a two-character string, in which the first character is ‘2’ and the second character is ‘²’. The second one contains digits, but it’s not described as a decimal number. Thanks to Unicode, you can create such strings either via the keyboard or using Unicode-entry syntax. Thus:

>>> s = '2²'    # or if you prefer, s = '2' + '\u00B2'

>>> s.isdigit()
True

>>> s.isnumeric()
True

>>> s.isdecimal()
False

Most of us, most of the time, can thus use these three methods interchangeably, with little chance of being mixed up. Once you start using all sorts of interesting Unicode-based numbers, things can get a bit weird and interesting.

I tried to determine whether there was any difference in speed between these methods, just in case, but after numerous tests with “%timeit” in Jupyter, I found that I was getting roughly the same speeds from all methods.

If you’re like me, and ever wondered how a language that claims to have “one obvious way to do it” can have multiple seemingly identical methods… well, now you know!

January 18, 2019 , bY reuven
Python

One of the first things that anyone learns in Python is (of course) how to print the string, “Hello, world.”  As you would expect, the code is straightforward and simple:

print('Hello, world')

And indeed, Python’s “print” function is so easy and straightforward to use that we barely give it any thought.  We assume that people know how to use it — and for the most part, for most of the things they want to do, that’s true.

But lurking beneath the surface of the “print” function is a lot of functionality, as well as some history (and even a bit of pain).  Understanding how to use “print” can cut down on the code you write, and generally make it easier for you to work with.

The basics

The basics are simple: “print” is a function, which means that if you want to invoke it, you need to use parentheses:

>>> print('hello')

hello

You can pass any type of data to “print”. Strings are most common, but you can also ints, floats, lists, tuples, dicts, sets, or any other object. For example:

>>> print(5)

5

or

>>> print([10, 20, 30])

[10, 20, 30]

And of course, it doesn’t matter whether the thing you’re trying to print is passed as a literal object, or referenced by a variable:

>>> d = {'a':1, 'b':2, 'c':3}>>> print(d)

{'a':1, 'b':2, 'c':3}

You can also put an expression inside of the parentheses; the value of the expression will be passed to “print”:

>>> print(3+5)
8

>>> print([10, 20] + [30, 40])
[10, 20, 30, 40]

Every object in Python knows how to display itself as a string, which means that you can pass it directly to “print”. There isn’t any need to turn things into strings before handing them to “print”:

print(str([10, 20, 30])    # unnecessary use of "str"

[10, 20, 30]

After “print” displays its output, it adds a newline.  For example:

>>> print('abc')
>>> print('def')
>>> print('ghi')
abc
def
ghi

You can pass as many arguments as you want to “print”, separated by commas. Each will be printed, in order, with a space between them:

>>> print('abcd', 'efgh', [10, 20, 30], 99, 'ijkl')

abcd efgh [10, 20, 30] 99 ijkl

We’ll see, below, how we can change these two default behaviors.

Inputs and outputs

If “print” is a function, then it must have a return value. Let’s take a look:

>>> x = print('abcd')
>>> type(x)
NoneType

In other words: “print” returns None, no matter what you print. After all, you’re not printing in order to get a return value, but rather for the side effect.

What about arguments to “print”?  Well, we’ve already seen that we can pass any number of arguments, each of which will be printed.  But there are some optional parameters that we can pass, as well.

The two most relevant ones allow us to customize the behavior we saw before, changing what string appears between printed items and what is placed at the end of the output.

The “sep” parameter, for example, defaults to ‘ ‘ (a space character), and is placed between printed items.  We can set this to any string, including a multi-character string:

>>> print('a', 'b', 'c', sep='*')
a*b*c

>>> print('abc', 'def', 'ghi', sep='***')
abc***def***ghi

>>> print([10, 20, 30], [40, 50, 60], [70, 80, 90], sep='***')
[10, 20, 30]***[40, 50, 60]***[70, 80, 90]

Notice that “sep” is placed between the arguments to “print”, not between the elements of each argument.  Thus in this third example, the ‘***’ goes between the lists, rather than between the integer elements of the lists.

If you want the arguments to be printed alongside one another, you can set “sep” to be an empty string:

>>> print('abc', 'def', 'ghi', sep='')
abcdefghi

Similarly, the “end” parameter defaults to ‘\n’ (newline), but can contain any string. It determines what’s printed after “print” is done.

For example, if you want to have some extra lines after you print something, just change “end” so that it has a few newlines:

>>> def foo():
        print('abc', end='\n\n\n')
        print('def', end='\n\n\n')
>>> foo()
abc


def


If, by contrast, you don’t want “print” to add a newline at the end of what you print, you can set “end” to be an empty string:

>>> def foo():
        print('abc', end='')
        print('def', end='')

>>> foo()
abcdef>>>

Notice how in the Python interactive shell, using the empty string to print something means that the next ‘>>>’ prompt comes after what you printed.  After all, you didn’t ask for there to be a newline after what you wrote, and Python complied with your request.

Of course, you can pass values for “end” that don’t involve newlines at all. For example, let’s say that you want to output multiple fields to the screen, with each field printed in a separate line:

>>> def foo():
        print('abc', end=':')
        print('def', end=':')
        print('ghi')

>>> foo()
abc:def:ghi

Printing to files

By default, “print” sends its data to standard output, known in Python as “sys.stdout”.  While the “sys” module is automatically loaded along with Python, its name isn’t available unless you explicitly “import sys”.

The “print” function lets you specify, with the “file” parameter, another file-like object (i.e., one that adheres to the appropriate protocol) to which you want to write. The object must be writable, but other than that, you can use any object.

For example:

>>> f = open('myfile.txt', 'w')

>>> print('hello')
hello
>>> print('hello???', file=f)
>>> print('hello again')
hello again
>>> f.close()

>>> print(open('myfile.txt').read())
hello???

In this case, the output was written to a file.  But we could also have written to a StringIO object, for example, which acts like a file but isn’t one.

Note that if I hadn’t closed “f” in the above example, the output wouldn’t have arrived in the file. That’s because Python buffers all output by default; whenever you write to a file, the data is only actually written when the buffer fills up (and is flushed), when you invoke the “flush” method explicitly, or when you close the file, and thus flush implicitly. Using the “with” construct with a file object closes it, and thus flushes the buffers as well.

There is another way to flush the output buffer, however: We can pass a True value to the “flush” parameter in “print”.  In such a case, the output is immediately flushed to disk, and thus written.  This might sound great, but remember that the point of buffering is to lessen the load on the disk and on the computer’s I/O system. So flush when you need, but don’t do it all of the time — unless you’re paid by the hour, and it’s in your interest to have things work more slowly.

Here’s an example of printing with and without flush:

>>> f = open('myfile.txt', 'w')
>>> print('abc', file=f)
>>> print('def', file=f)
>>> print(open('myfile.txt').read())  # no flush, and thus empty file

>>> print('ghi', file=f, flush=True)  
>>> print(open('myfile.txt').read())  # all data has been flushed to disk
abc
def
ghi

You might have noticed a small inconsistency here: “print” writes to files, by default “sys.stdout”. And if we don’t flush or close the file, the output is buffered.  So, why don’t we have to flush (or close, not that this is a good idea) when we print to the screen?

The answer is that “sys.stdout” is treated specially by Python. As the Python docs say, it is “line buffered,” meaning that every time we send a newline character (‘\n’), the output is flushed.  So long as you are printing things to “sys.stdout” that end with a newline — and why wouldn’t you be doing that? — you won’t notice the buffering.

Remember Python 2?

As I write this, in January 2019, there are fewer than 12 months remaining before Python 2 is no longer supported or maintained. This doesn’t change the fact that many of my clients are still using Python 2 (because rewriting their large code base isn’t worthwhile or feasible).  If you’re still using Python 2, you should really be trying to move to Python 3.

And indeed, one of the things that strikes people moving from Python 2 to 3 would be the differences in “print”.

First and foremost, “print” in Python 2 is a statement, not an expression. This means that the parentheses in 2 are optional, while they’re mandatory in 3 — one of the first things that people learn when they move from 2 to 3.

This also means that “print” in Python 2 cannot be passed to other functions. In Python 3, you can.

Python 2’s “print” statement didn’t have the parameters (or defaults) that we have at our disposal.  You wanted to print to a file other than “sys.stdout”?  Assign it to “sys.stdout” to use “print” — or just write to the file with the “write” method for files.  You wanted “print” not to descend a line after printing?  Put a comma at the end of the line.  (Yes, really; this is ugly, but it works.)

What if you’re working in Python 2, and want to get a taste of Python 3’s print function?  You can add this line to your code:

from __future__ import print_function

Once you have done so, Python 3’s “print” function will be in place.

Now I know that Python 3 is no longer in the future; indeed, you could say that Python 2 is in the past. But for many people who want to transition or learn how to do it, this is a good method. But watch out: If you have calls to “print” without parentheses, or are commas to avoid descending a line, then you’ll need to do more than just this import.  You will need to go through your code, and make sure that it works in this way. So while that might seem like a wise way to to, it’s only the first step of a much larger transition from 2 to 3 that you’ll need to make.

Enjoyed this article?  Join more than 11,000 other developers who receive my free, weekly “Better developers” newsletter. Every Monday, you’ll get an article like this one about software development and Python:



 

January 15, 2019 , bY reuven
Python

Weekly Python Exercise logoIf you want to join this month’s cohort of Weekly Python Exercise, you’d better act fast: Registration ends today!

  • If you feel stuck after having taken a Python course or read a book, and don’t know how to improve,
  • If you want to get real-world practice problems that’ll help you with your career,
  • If you want to stop searching Stack Overflow each time you have to use Python,
  • If you want to benefit from a community of learners who assist one another

Then Weekly Python Exercise is a great way to go.

Click here to join Weekly Python Exercise.

Questions?  You can always e-mail me (reuven@lerner.co.il) or hit me up on Twitter (@reuvenmlerner).  Or you can watch the Q&A Webinar I held last night, answering questions from others interested in joining this cohort:

And don’t forget that I offer discounts for students, pensioners/retirees, and people living outside of the world’s 30 wealthiest countries.

Hundreds of other developers have improved their Python skills with WPE in the last 18 months.  Improve your career, stop feeling stuck, and stop searching Stack Overflow, and improve your Python fluency!