Python makes it easy to write functions. For example, I can write:
def hello(name):
return f'Hello, {name}!'
I can then run the function with:
hello('world')
which will then, not surprisingly, return the string
'Hello, world'
Here’s a question that doesn’t come up much: How does Python assign the string argument ‘world’ to the parameter “name”? That is, how does Python know that the value “world” should be assigned to the variable “name”?
You might be thinking, “It’s obvious. I mean, there’s only one value, and there’s only one variable. How else could it work?”
But of course, programming languages don’t work based on what we think is obvious. They have rules, and follow those rules to the letter. So, what are the rules for assigning arguments to parameters?
It turns out that this question isn’t as trivial as it seems. And that’s because while function parameters in Python don’t indicate what types of values they can accept, they do tell Python what types of arguments they can accept — that is, the ways in which they can (or cannot) be assigned arguments when the function is called. (And yes, type hints sorta kinda let us indicate what types of values can be assigned to parameters, but the Python language itself ignores these.)
In this blog post, I want to review the seven (!) different types of parameters, and how Python assigns arguments to them.
Positional vs. keyword arguments
Before we can talk about parameters, we have to talk about arguments. And yes, most developers will confuse these two terms, or use them interchangeably. Simply put, an argument is a value that is passed to a function, whereas a parameter is a variable to which we assign a value. Parameters are local variables, and work just like all other local variables in Python.
When you call a function, you can pass values as two different types of arguments:
- Positional arguments, where the position of the argument in the call indicates which parameter should get its value
- Keyword arguments, which always have the form name=value, tell the receiving function which parameter (name) should be assigned the argument (value).
That’s it: There are no other types of arguments in Python! Everything is either positional or keyword. Parameters can indicate how they’re willing to be assigned arguments.
Simple parameters
The simplest, and most recognizable, parameters in Python are the ones without any special decorations. For example, the “hello” function from the top of this blog post has a single, simple parameter:
def hello(name):
return f'Hello, {name}!'
I can call this function with a single positional argument:
hello('world')
Or I can call this function with a single keyword argument:
hello(name='world')
In the first case, Python sees that we have passed a single positional argument. It checks the function’s definition to see how many arguments we will accept (__code__.co_argcount in the function object) and then grabs the parameter’s name (__code__.co_varnames in the code object). Python thus knows that argument #1 should be assigned to parameter #1 — meaning, “world” should be assigned to “name”.
In the second case, Python sees that we have passed a single keyword argument, with a name “name” and a value “world”. It checks (again, with __code__.co_argcount and __code__.co_varnames) whether there is a parameter called “name”. Once it discovers that there is, it assigns “world” to the variable “name”, and all is good.
What happens if we pass the wrong number of positional arguments? Python notices this, and raises an exception, telling us how many arguments the function expects to get, and how many we actually passed. (We’ll get back to this in a little bit.) For example:
>>> hello('a', 'b', 'c')
Traceback (most recent call last):
File "", line 1, in
TypeError: hello() takes 1 positional argument but 3 were given
What happens if we pass keyword arguments for parameters that don’t exist? Python notices this as well, and raises an exception. For example:
>>> hello(a=10, b=20, c=30) Traceback (most recent call last): File "", line 1, in TypeError: hello() got an unexpected keyword argument 'a'
Multiple parameters
What if a function has multiple parameters? Then we can invoke it with positional arguments, keyword arguments, or even a mixture of these. For example, let’s take the following function:
def calc(first, op, second):
if op == '+':
return first + second
elif op == '-':
return first - second
else:
return f'Operator {op} is unsupported'
I can call this function in a variety of ways:
- Only with positional arguments, as in “calc(10, ‘+’, 3)”
- Only with keyword arguments, as in “calc(first=10, op=’+’, second=3)”
- Keyword arguments in a different order, as in “calc(second=3, first=10, op=’+’)”.
- A mixture of positional and keyword arguments, as in “calc(10, op=’+’, second=3)”
This last example is perhaps the most interesting of the bunch. That’s because Python always allows you to mix positional and keyword arguments, so long as all of the positional arguments come before all of the keyword arguments. Here’s what happens if you get this wrong:
>>> calc(first=10, op='+', 3)
File "", line 1
calc(first=10, op='+', 3)
^
SyntaxError: positional argument follows keyword argument
If a function has these sorts of simple parameters, then we can call the function with any combination of positional and keyword arguments, so long as the number of arguments matches the number of parameters and all of the positional arguments come before the keyword arguments.
Parameters with defaults
Sometimes, we want to give a parameter a default value, so that the caller doesn’t need to provide an argument. We can do this with a syntax that’s identical to that of keyword arguments, name=value, except that this is done in the function definition, not in the function call. For example, we can rewrite our “calc” function as follows:
def calc(first, op='+', second=0):
if op == '+':
return first + second
elif op == '-':
return first - second
else:
print(f'Operator {op} is unsupported')
The above function is identical to the previous version, except that we’ve now indicated that the caller is only required to pass one argument, which will be assigned to the parameter “first”. Both “op” and “second” are now optional, because if we don’t pass them any arguments, they’ll get the default values “+” and 0.
This is possible because when we create our function, Python creates an attribute, “__defaults__”, on the newly created function object. __defaults__ is a tuple containing all of the default values. When we call the function, it checks __code__.co_argcount to find out how many arguments we’re supposed to pass — and if we’re missing one or more, Python grabs the missing one(s) from __defaults__.
In order to make this work, parameters with defaults must be defined after those that lack defaults. That makes it possible for Python to figure out which parameters didn’t get values from the caller, and thus need to get values from __defaults__.
For example, __defaults__ in the “calc” function is (‘+’, 0). So if I invoke “calc(5)”, Python sees that we need to pass three arguments. But we’ve only passed one, so it needs to retrieve the final two values from calc.__defaults__ and then assign those values to “op” and “second”. After it does that, it invokes the function — whose body is blissfully unaware of how this happened.
It’s important to point out that the default values are set when the function is defined, and thus compiled. For this reason, you can get into big, big trouble if you use a mutable default, such as a list. For example:
def add_one(x=[]):
x.append(1)
return x
You might think that the default value in the above function means: If I call “add_one” without any arguments, then x will be assigned to an empty list. But no! The list is created when we define the function, and stored away in __defaults__. This means that every time we call the function without any arguments, Python will grab the list from __defaults__ and assign it to x. The function will modify the list, which not only affects x, but also affects the list as stored in the __defaults__ tuple. In other words, our empty list will not be very empty after several calls to the function.
The moral of this story is: Never use mutable default values in a function. Code checkers like “pylint” will notice this for you, but once you realize just how bad this can be, you will likely avoid it without any help.
So far, both of these parameters types can be assigned arguments via either positional or keyword arguments. Now let’s shake things up a bit, though, and talk about the weirder and more unusual parameter types.
*args (aka: all unclaimed positional arguments)
Many people are familiar with the term “*args”, and even use it a bit — but I’ve found that they often don’t understand just what it’s for, or how to use it.
If you have a parameter whose name is preceded by “*”, then its type will be a tuple, and it will contain all of the positional arguments that weren’t assigned to another parameter. Traditionally, this parameter is called “*args”, and pronounced “splat-args,” on the assumption that the “*” sign looks something like a squashed bug. You can call this parameter whatever you want, though — and I often do.
Once your function has *args as a parameter, there is no maximum number of positional arguments that it can take. Any extras, as it were, will be placed into *args. For that reason, *args must be the last parameter that can take positional arguments. (We’ll see some others which can only take keyword arguments in a little bit.) If there are any parameters preceding *args, then *args will be assigned those arguments that weren’t grabbed by the other parameters.
I should note that there isn’t really a technical advantage to use *args. You could, of course, just have a single parameter that takes a list or tuple. But there’s something that feels natural about passing a bunch of values as individual arguments, rather than passing a list or tuple. For example, if I have a function that sums numbers, then it might make more sense to pass he numbers as separate arguments than to pass a list. If I have a function that performs a network security check on IP addresses, then I could take the addresses as positional arguments, rather than as a list. And if I have a function that summarizes data from files, it would make sense to have the function take filenames (strings), rather than a list of filenames.
For example, we can write a number-summing function as follows:
def mysum(numbers):
total = 0
for one_number in numbers:
total += one_number
return total
To run the above function, I need to pass an iterable of numbers, as in “mysum([10, 20, 30])”. That’s because the function, as defined, takes a single argument (either positional or keyword). But if I change the function definition ever so slightly:
def mysum(*numbers):
total = 0
for one_number in numbers:
total += one_number
return total
Now the function expects to get only positional arguments, and any number of them. The parameter (i.e., variable) “numbers” will now be a tuple, containing whatever values were passed to the function. I can now call:
- mysum(10, 20, 30)
- mysum(1,2,3,4,5)
- mysum() # no elements, returns 0
- mysum(*[10, 20 ,30]) # a bit advanced — turns a list into individual arguments!
What if the function has other parameters that accept positional arguments? Those must be before *args. And we can have as many of those as we like. For example:
def myfunc(a, b, *args):
return f'{a=}, {b=}, {args=}'
If I call “myfunc(10, 20, 30, 40, 50)”, then we’ll see that a is 10, b is 20, and args is the tuple (30, 40, 50).
The typical use for *args is in a “for” loop, list comprehension, or similar iteration. If your function accepts *args but then pulls out index 0, 1, or 2 specifically, then you are probably using it wrong. Similarly, you probably don’t want to be calling “len” on the tuple. The idea is that you’ll do the same thing with all of the arguments, no matter how many (or few) there might be.
What happens if I want my function to have a parameter with a default, and then take *args? For example:
def myfunc(a, b=10, *args):
return f'{a=}, {b=}, {args=}'
If I run “myfunc(5,6,7,8,9)”, then it’s pretty obvious that a will be 5, b will be 6, and args will be the tuple (7,8,9). But what if I want to pass a value for “a”, pass additional positional arguments to “args”, and have the default value of “b” in place?
Answer: You can’t. There isn’t any way to do that. Because along the way to *args, you have to give “b” a value. (We’ll see in a bit how we can use keyword-only parameter to accomplish this.)
You might be thinking: Of course there’s a way to do this! I can just call the function, passing “args” as a keyword argument. Then I can sidestep “b” and give a value to “args”. But that won’t work, because you would be assigning to “args” as a keyword argument, and args (by definition) only takes positional values. You cannot use “args” as the name in a keyword argument; Python will tell you that it’s not aware of such a parameter.
**kwargs (aka: all unclaimed keyword arguments)
Just as *args allows a function to accept as many positional arguments as we might want to pass, **kwargs allows a function to accept as many keyword arguments as we might like. As with *args, the name “kwargs” is traditional but not mandatory; you can use any name you want. The variable “kwargs” is a dict in which the keys are strings, and the values are whatever was passed in the keyword arguments For example:
def myfunc(a, b=10, **kwargs):
return f'{a=}, {b=}, {kwargs=}'
If I call “myfunc(5, 6, 7, 8)”, then I’ll get an error — because the function only accepts up to two positional arguments. All the rest must be keyword arguments! So I can try “myfunc(5,6, x=100, y=200)” and then we’ll see the following output:
"a=5, b=6, kwargs={'x': 100, 'y': 200}"
Notice that our keyword arguments (x and y) were turned into strings. Both 100 and 200 remained as integers, as they were passed.
Also notice that we don’t have the problem with “b” that we did before, with *args. We can just call “myfunc(5, x=100, y=200)”, and we’ll get:
"a=5, b=10, kwargs={'x': 100, 'y': 200}"
See? We got the default value for “b”, because there weren’t any positional arguments to override it.
Keyword-only parameters
We saw earlier that we cannot have a parameter with a default before our *args parameter. Fortunately, Python 3 introduced a new kind of parameter: The keyword-only parameter. Any parameter that comes after *args can only be set with keyword arguments.
There are actually two types of keyword-only parameters: If you define it as name=value, then the parameter has a default value, and doesn’t need to be specified by the caller. But if you just include the parameter name, without a default value, then the keyword argument is mandatory, and calling the function without it will result in an error.
For example:
def myfunc(a, *args, b, c=10):
…: return f'{a=}, {b=}, {c=}, {args}'
If I call it with “myfunc(5,6,7,8,9)”, then we get the following error:
myfunc() missing 1 required keyword-only argument: 'b'
I can then call it with “myfunc(5,6,7,8,9,b=22)” and we get:
'a=5, b=22, c=10, (6, 7, 8, 9)'
Positional-keyword separator
What if you want to have keyword-only parameters, but you don’t want the function to accept *args? Or what if you want your function to only accept keyword arguments (i.e., no positional arguments)? How can you do that?
If you put * in the list of parameters, then all parameters following it are keyword only. In other words, we can define:
def hello(*, first, last):
return f'Hello, {first} {last}'
We can now call this function as “hello(first=’Reuven’, last=’Lerner’)”. But we cannot call it with any positional arguments.
You can, of course, put the “*” after one or more other parameters. Those preceding parameters can accept either positional or keyword arguments.
Positional-only parameters
The newest kind of parameter is the positional-only parameter. Such parameters, as you can gather from their names, cannot be set with keyword arguments. They are specified by putting a “/” (slash) in the parameter list. Any parameter that comes before a “/” is considered positional only. For example:
def add(a, b, /):
return a + b
If I call it with “add(10, 3)”, then I get 13. But if I call it with “add(a=10, b=3)”, then I get the following error:
TypeError: add() got some positional-only arguments passed as keyword arguments: 'a, b'
Now, why would you want positional-only parameters? It’s to avoid clashes with keyword arguments. What do I mean by that?
Let’s assume that you want to write a function that takes any keyword arguments and writes them to a file, in the style of a configuration file – with each key-value pair turned into “key:value” in the file. For example:
def write_config(filename, **kwargs):
with open(filename, 'w') as f:
for key, value in kwargs.items():
f.write(f'{key}:{value}\n' )
We can call it as follows:
write_config('myconfig.txt', a=10, b=20, c=30)
But what happens if one of the configuration keys is called “filename”? Let’s try it:
write_config('myconfig.txt', filename='abc', a=10, b=20, c=30)
We get an exception:
TypeError: write_config() got multiple values for argument 'filename'
This is why we have positional-only parameters, so that the keyword arguments don’t clash with the positional arguments. If I redefine my function such that “filename” is positional-only:
def write_config(filename, /, **kwargs):
with open(filename, 'w') as f:
for key, value in kwargs.items():
f.write(f'{key}:{value}\n' )
Now we won’t have any problems calling write_config with a “filename” keyword argument, because Python won’t see it as having been set twice, and kwargs only looks at keyword arguments:
write_config('myconfig.txt', filename='abc', a=10, b=20, c=30)
Summary
While Python doesn’t have parameter for different types of values, it does have different sorts of parameters, all of which tell Python whether they will accept positional arguments, keyword arguments, or both. Defining your parameters in the right way will make your code clearer, and make it easier for people who want to call your functions to know how they should do so.
Python allows you to define your function with any or all of the following parameter types, in the following order:
- Positional-only parameters, before a /.
- Mandatory parameters, which can be passed either positional or keyword.
- Optional parameters (thanks to a default value), which can be passed either positional or keyword.
- *args, a tuple that contains all otherwise unclaimed positional arguments. You can also put a * here. to indicate that following this point, all parameters are for keyword arguments.
- Mandatory keyword-only parameters. The parameter must be named as a keyword argument in order for the function to run.
- Optional keyword-only parameters. If the parameter isn’t named in a keyword argument, Python gives it a default value.
- **kwargs, a dictionary containing all otherwise unclaimed keyword arguments.
Want to learn more? My free, weekly “Better developers” newsletter brings you this kind of article and information to your inbox every Monday. Check it out at https://BetterDevelopersWeekly.com/.
I liked the overview your article provided! Thanks for that.