1 year ago

#303077

test-img

CutOnBumInBand

When type hinting python functions, why are `*args` and `**kwargs` trearted differently?

I'm learning my way around type hints in modern python, specifically how to express the type of functions and their parameters.

If I have a function f and I know nothing about it, I can write its type as Callable (or equivalently, Callable[..., Any]); if I know its return type, I can specify Callable[..., ReturnType], and finally if I know everything about it, I can write Callable[[Arg1Type, Arg2Type, \ldots], ReturnType].

What do I do if I only know some of the argument types, but still want to enforce that contract? This answer on stackoverflow suggests that a useful approach is to create a Protocol with a suitable example of what the call should look like.

Doing that lets me specify a type like "A function that takes an integer as the first parameter, and then an arbitrary number of positional arguments, and returns an integer". And indeed, the following type checks:

from typing_extensions import Protocol

class Distribution(Protocol):
    def __call__(self, x: int, *args) -> int:
        ...

def f1(x: int, s: str) -> int:
    ...

def f2(x: int, a: float, b: float) -> int:
    ...

distribution: Distribution = f1
distribution: Distribution = f2

That's great!

But what if I want to express the type of a function that takes an integer as its first argument and then an arbitrary number of keyword arguments, and returns an integer? The obvious approach would be to change the protocol to

class Distribution(Protocol):
    def __call__(self, x: int, **kwargs) -> int:
        ...

Unfortunately this doesn't work; the type checker complains that **kwargs has no corresponding parameter in either f1 or f2. Now, I can just about convince myself that either result (error or not) is valid:

  • Obivously the code can't work! The protocol specifies that the function should take an arbitrary number of keyword arguments, not some specific ones. Indeed, changing the signature of f1 to f1(x: int, s: str, **_) makes everything type check. In this view, Callable[[a, arg1], a] is not a valid subtype of Callable[[a, **kwargs], a].

  • Obviously the code should work! I specify that I want a specific argument, which is present, and then any further arguments are handled in the rest of the *args tuple. Both f1 and f2 match this spec, so there are no issues. In this view, Callable[[a, arg1], a] is a valid subtype of Callable[[a, *args], a].

What I cannot understand, is why one example works and the other doesn't.

Does anyone know?

Edit: This appears to be a bug in pyright since the following also passes with no complaint

from typing_extensions import Protocol

class Distribution(Protocol):
    def __call__(self, x: int, *args) -> int:
        ...

def no_parameters(x: int) -> int:
    ...

distribution: Distribution = no_parameters

Looking at their github it might have been fixed five hours ago, but I haven't tested it yet.

python

types

pyright

0 Answers

Your Answer

Accepted video resources