The Python type system is pretty bad, but it's still 100x better than not using types. We are heavy users of the (Rust) type system at Svix, and it's been a godsend. I wrote about it here https://www.svix.com/blog/strong-typing-hill-to-die-on/
We also use Python in some places, including the shitty Python type-system (and some cool hackery to make SQLAlchemy feel very typed and work nicely with Pydantic).
Looking at that blog post, I find it illustrative in how people who like strong types and people who dislike strong types are addressing different form of bugs. If the main types of issues comes from bugs like 1 + "2" == 12", then strong types is a big help. It also enables many developers who spend the majority of time in a programming editor to quickly get automatic help with such bugs.
The other side is those people who do not find those kind of bugs annoying, or they simply don't get hit by such bugs at a rate that is high enough to warrant using a strong type system. Developers who spend their time prototyping in ipython also get less out of the strong types. The bugs that those developers are concerned about are design bugs, like finding out why a bunch of small async programs reading from a message buss may stall once every second Friday, and where the bug may be a dependency of a dependency of a dependency that do not use a socket timeout. Types are similar not going to help those who spend the wast majority of time on bugs where someone finally says "This design could never have worked".
If it’s 100x better than no types, then probably 10x better than C++ type system. It takes some time to unlearn using dicts everywhere, but then namedtuples become your best friend and noticeably improve maintainability. Probably the only place where python type system feels inadequate is describing json-like data near the point of its (de)serialization.
> (and some cool hackery to make SQLAlchemy feel very typed and work nicely with Pydantic).
Sounds interesting. Can you elaborate on the cool hackery? We introduced SQLModel recently but struggle in a few cases (e.g. multi-level joins). Do you know reference projects for SQLAlchemy and pydantic?
Type hints are nice, until you have to interact with a library that isn't type-hinted, and then it very quickly becomes a mess.
I don't know how other IDEs behave, but VScode + the Python extensions try to infer the missing hints and you end up with beauties such as `str | None | Any | Unknown`, which of course are completely meaningless.
Even worse, the IDE marks as an error some code that is perfectly correct, because it somehow doesn't match those nonsensical hints. And so it gives you the worst of both worlds: a lot of false positives that you quickly learn to ignore, dooming the few actual type errors to irrelevance, because you'll ignore them anyways until they blow up at runtime, just as it'd happen without typehints.
> Type hints are nice, until you have to interact with a library that isn't type-hinted, and then it very quickly becomes a mess.
Whenever I find myself in that situation, I usually write a typing stub for the parts that I use from that library (example: [0]) and then let `mypy_path` point to that directory [1].
VS Code will then pick up the hints from those stubs.
>I don't know how other IDEs behave, but VScode + the Python extensions try to infer the missing hints and you end up with beauties such as `str | None | Any | Unknown`, which of course are completely meaningless.
Are they correct? If they're correct (even though they are a superset of the actual intended type) then what's the problem?
At worst, it's like not having type checks for that particular package.
They are verbose but correct. I've caught some errors this way.
I usually don't think of None as a potential return value (= voids in C) but the LSP code analysis usually picks up on code paths that don't return a value.
I don't find Python's typing valuable for Jupyter type explorations, but they're immensely valuable for catching little issues in production code.
For example in mypy the default is to not check procedures, which have no argument type annotations and no return type annotation. That gets rid of your whole problem of untyped library, if you have a wrapper procedure.
If VSCode still highlights it, then it is time to configure VSCode properly.
I believe VSCode by default uses pyright which is fast but shitty in that it gives a lot of false positives. If you want the most correct typing experience, use mypy. Even then you may need a config.
I get what you need, yet I find these cases aren't all that often, and when it happens it doesn't bother me as I quickly recognize where the type system is somewhat failing and either ignore it or add a type hint.
But maybe if you have a codebase with a lot of magic of certain libraries, you experience is different. I also don't really depend on the typing treat it the same as C# or Java.
Worst of both worlds is right. I came back to a Python project with a couple of critical but untyped dependencies recently after writing mostly Rust, and to clear up a large number of these (particularly “type is partially unknown”) I had the choice between lots of purely type-checking ceremony (`typing.cast`) or going without.
The third option here is writing type stubs for the library, which you can sometimes find community versions of as well. They’re not too time consuming to write and generally work well enough to bridge the gap
Yeah, I think this may be a good option when actively working on a project. Sadly at the moment, it's mostly a case of "I just need to make a couple of bug fixes in this old project, why is my editor shouting at me?"
It's only a personal side project and I have a good handle on the untyped modules in question, so in the end I suppressed most of the errors with `# type:ignore` and friends.
I'd reconsider that if I was doing more than the odd bug fix on the project. I still like Python, and started using type hints early, but there's enough added friction to make me question using them in the future.
> In writing this it occurs to me that I do often know that I have distinct types (for example, for what functions return) and I shouldn't mix them, but I don't want to specify their concrete shape as dicts, tuples, or whatever. [...] Type aliases are explicitly equivalent to their underlying thing, so I can't create a bunch of different names for eg typing.Any and then expect type checkers to complain if I mix them.
It sounds to me like you're describing the NewType pattern which is just slightly farther down the page you linked in the article.
The logic of type hint is not bad but sadly I think that type hint are making python source code messy and unreadable.
I'm missing a lot simple functions with explicit argument names and docstrings with arguments types and descriptions clearly but discreetly documented.
It was one big strength of Python to have so simple and clean code without too much boilerplate.
Also, I have the feeling that static typing extremist are trying to push the idea that type hinting allows to ensure to not mix types as it would be bad. But from my point of view the polymorphic and typing mixing aspect is a strong force of Python.
Like having dictionaries that are able to hold whatever you want is so incredible when you compare to trying to do the equivalent in Java for example.
One part where I find type hint to be wonderful still is for things like pydantic and dataclasses!
> Like having dictionaries that are able to hold whatever you want is so incredible when you compare to trying to do the equivalent in Java for example.
Can't you just make a dictionary of objects, same as in C#? Except that in C#, if you really want to, you can also use `dynamic` to get python-like behavior.
Otherwise, generally speaking, in a strongly typed language you want to figure out what those objects have in common and put that inside an interface. If you can't modify those objects just slap an adapter pattern on top.
The result is a dictionary of objects that adhere to a specific interface, which defines all the properties and procedures that are relevant to the domain and the problem.
This makes thinking about the problem much easier from a type theoretical perspective because it lets you abstract away the concrete details of each object while preserving the fundamental aspects that you care about.
I guess that it takes two different mindsets to in order to appreciate the pros and cons of dynamic vs static programming. There are certainly many pros for dynamic programming, but I'm more comfortable thinking about problems in generic terms where every relation and constraint is laid bare in front of me, one level removed from the actual implementation.
Type hinting in python is a bit of a sticky plaster.
We have pyre enforcement at work, the problem is, that it has been gradually turned on over time, so some stuff is pyre compliant (or just strategically ignored) and some stuff isnt, so when you open some old code to do something, you have a million errors to deal with.
That would be fine if types were enforceable. in runtime type hinting does shit all.
I would dearly love a "strict" mode where duck typing is turned off and variable are statically typed. However I suspect that will never happen, even though it'd speed up a load of stuff if done correctly (type inference happens a lot)
I suspect to use type hints properly, I'd need to think a bit more C-like and create dataclasses as types to make things more readable, rather than using Dict[str,int] or what ever.
There are some other ways of expressing names for types, once you start using typing. There are typevars, enums and using the "|" to separate options, there are TypedDict, NamedTuple, Union, Literal, Optional, and probably more. Not everything needs to be a dataclass.
The problem (in my opinion) is that Python gives you the tools (and perhaps even encourages you) to write code that would benefit from typing.
It's perfectly feasible to write maintainable, well-designed code in a dynamic language. I've worked with some extremely robust and ergonomic Clojure codebases before, for example. However, in Clojure, the language pushes you into its own "pit of success".
> yet another Python thing I'd have to try to keep in my mind despite it being months since I used them last.
Typing hints are also a moving target: they have changed, sometimes significantly, on every minor Python release since they came to be.
The `Optional` type came and went (being replaced by the new Union syntax.) Type classes are usually born in the `typing` module, then some of them get moved to `collections`, `abc`, or `collections.abc`. Some types have even been moved several times. `TypeAlias` came and went (replaced by `type`). `List` became `list`. `Tuple` became `tuple`. Forward declarations require a workaround by using string literals. But in Python 3.14, they no longer do, and the workaround will become deprecated.
I'm an evangelist for static type checking, and I never write a Python function signature without complete typing annotations. I also think that typing annotations in Python are evolving in a good way. However, for long-lived Python scripts, I highly recommend that developers are aware of, and factor in, the effort necessary to keep up with the evolution of static typing across Python versions. That effort spent on migrating is going to come on top of the mental load that typing hints already add to your plate.
> PPS: I think my ideal type hint situation would be if I could create distinct but otherwise unconstrained types for things like function arguments and function returns, have mypy or other typing tools complain when I mixed them
Cleaner/safer function args and return types is a common motivation for dataclass. has benefits over many/complex args besides typing too.
I typehint the stuff that is easy. My observations about typehinting in Python track the 80:20 rule or even a 90:10 rule. You get about 80% benefit for typhinting the easy 20%.
this is not a just a python problem but a problem in many dynamic languages.
switching to the typed variants whether typecript, python type-hints, mypy etc will force you to do the dance to make the compiler happy instead of working on code.
which is why for me - JSDoc is really good - it uses types for documentation and ends there.
Absolutely. The main problem with python typing is that checking types is optional. A dialect with mandatory types (with inference) and runtime/load-time checking would be great.
It’s kind of cool that type hints can be reflected on to do things (see Pydantic). Other than that I find it pretty cumbersome to use in practice, coming from TypeScript. Semi-relatedly I also dislike Python’s different ways to access objects/dicts, it feels arbitrary and cumbersome.
The Python type system is pretty bad, but it's still 100x better than not using types. We are heavy users of the (Rust) type system at Svix, and it's been a godsend. I wrote about it here https://www.svix.com/blog/strong-typing-hill-to-die-on/
We also use Python in some places, including the shitty Python type-system (and some cool hackery to make SQLAlchemy feel very typed and work nicely with Pydantic).
Looking at that blog post, I find it illustrative in how people who like strong types and people who dislike strong types are addressing different form of bugs. If the main types of issues comes from bugs like 1 + "2" == 12", then strong types is a big help. It also enables many developers who spend the majority of time in a programming editor to quickly get automatic help with such bugs.
The other side is those people who do not find those kind of bugs annoying, or they simply don't get hit by such bugs at a rate that is high enough to warrant using a strong type system. Developers who spend their time prototyping in ipython also get less out of the strong types. The bugs that those developers are concerned about are design bugs, like finding out why a bunch of small async programs reading from a message buss may stall once every second Friday, and where the bug may be a dependency of a dependency of a dependency that do not use a socket timeout. Types are similar not going to help those who spend the wast majority of time on bugs where someone finally says "This design could never have worked".
If it’s 100x better than no types, then probably 10x better than C++ type system. It takes some time to unlearn using dicts everywhere, but then namedtuples become your best friend and noticeably improve maintainability. Probably the only place where python type system feels inadequate is describing json-like data near the point of its (de)serialization.
> (and some cool hackery to make SQLAlchemy feel very typed and work nicely with Pydantic).
Sounds interesting. Can you elaborate on the cool hackery? We introduced SQLModel recently but struggle in a few cases (e.g. multi-level joins). Do you know reference projects for SQLAlchemy and pydantic?
Can you give some examples of how the Python type system is disappointing you?
Type hints are nice, until you have to interact with a library that isn't type-hinted, and then it very quickly becomes a mess.
I don't know how other IDEs behave, but VScode + the Python extensions try to infer the missing hints and you end up with beauties such as `str | None | Any | Unknown`, which of course are completely meaningless.
Even worse, the IDE marks as an error some code that is perfectly correct, because it somehow doesn't match those nonsensical hints. And so it gives you the worst of both worlds: a lot of false positives that you quickly learn to ignore, dooming the few actual type errors to irrelevance, because you'll ignore them anyways until they blow up at runtime, just as it'd happen without typehints.
> Type hints are nice, until you have to interact with a library that isn't type-hinted, and then it very quickly becomes a mess.
Whenever I find myself in that situation, I usually write a typing stub for the parts that I use from that library (example: [0]) and then let `mypy_path` point to that directory [1].
VS Code will then pick up the hints from those stubs.
[0]: https://github.com/claui/itchcraft/blob/5ca04e070f56bf794c38...
[1]: https://github.com/claui/itchcraft/blob/5ca04e070f56bf794c38...
>I don't know how other IDEs behave, but VScode + the Python extensions try to infer the missing hints and you end up with beauties such as `str | None | Any | Unknown`, which of course are completely meaningless.
Are they correct? If they're correct (even though they are a superset of the actual intended type) then what's the problem?
At worst, it's like not having type checks for that particular package.
They are verbose but correct. I've caught some errors this way.
I usually don't think of None as a potential return value (= voids in C) but the LSP code analysis usually picks up on code paths that don't return a value.
I don't find Python's typing valuable for Jupyter type explorations, but they're immensely valuable for catching little issues in production code.
For example in mypy the default is to not check procedures, which have no argument type annotations and no return type annotation. That gets rid of your whole problem of untyped library, if you have a wrapper procedure.
If VSCode still highlights it, then it is time to configure VSCode properly.
I believe VSCode by default uses pyright which is fast but shitty in that it gives a lot of false positives. If you want the most correct typing experience, use mypy. Even then you may need a config.
I get what you need, yet I find these cases aren't all that often, and when it happens it doesn't bother me as I quickly recognize where the type system is somewhat failing and either ignore it or add a type hint.
But maybe if you have a codebase with a lot of magic of certain libraries, you experience is different. I also don't really depend on the typing treat it the same as C# or Java.
Worst of both worlds is right. I came back to a Python project with a couple of critical but untyped dependencies recently after writing mostly Rust, and to clear up a large number of these (particularly “type is partially unknown”) I had the choice between lots of purely type-checking ceremony (`typing.cast`) or going without.
The third option here is writing type stubs for the library, which you can sometimes find community versions of as well. They’re not too time consuming to write and generally work well enough to bridge the gap
Yeah, I think this may be a good option when actively working on a project. Sadly at the moment, it's mostly a case of "I just need to make a couple of bug fixes in this old project, why is my editor shouting at me?"
What did you end up choosing & why?
It's only a personal side project and I have a good handle on the untyped modules in question, so in the end I suppressed most of the errors with `# type:ignore` and friends.
I'd reconsider that if I was doing more than the odd bug fix on the project. I still like Python, and started using type hints early, but there's enough added friction to make me question using them in the future.
I imagine on big projects the benefit is clearer.
Thanks for sharing!
Asking because I was really, really annoyed by the non-helpfulness of the type hints in practice, contrary to the theory.
> In writing this it occurs to me that I do often know that I have distinct types (for example, for what functions return) and I shouldn't mix them, but I don't want to specify their concrete shape as dicts, tuples, or whatever. [...] Type aliases are explicitly equivalent to their underlying thing, so I can't create a bunch of different names for eg typing.Any and then expect type checkers to complain if I mix them.
It sounds to me like you're describing the NewType pattern which is just slightly farther down the page you linked in the article.
https://docs.python.org/3/library/typing.html#newtype
The logic of type hint is not bad but sadly I think that type hint are making python source code messy and unreadable.
I'm missing a lot simple functions with explicit argument names and docstrings with arguments types and descriptions clearly but discreetly documented.
It was one big strength of Python to have so simple and clean code without too much boilerplate.
Also, I have the feeling that static typing extremist are trying to push the idea that type hinting allows to ensure to not mix types as it would be bad. But from my point of view the polymorphic and typing mixing aspect is a strong force of Python.
Like having dictionaries that are able to hold whatever you want is so incredible when you compare to trying to do the equivalent in Java for example.
One part where I find type hint to be wonderful still is for things like pydantic and dataclasses!
> Like having dictionaries that are able to hold whatever you want is so incredible when you compare to trying to do the equivalent in Java for example.
Can't you just make a dictionary of objects, same as in C#? Except that in C#, if you really want to, you can also use `dynamic` to get python-like behavior.
Otherwise, generally speaking, in a strongly typed language you want to figure out what those objects have in common and put that inside an interface. If you can't modify those objects just slap an adapter pattern on top.
The result is a dictionary of objects that adhere to a specific interface, which defines all the properties and procedures that are relevant to the domain and the problem.
This makes thinking about the problem much easier from a type theoretical perspective because it lets you abstract away the concrete details of each object while preserving the fundamental aspects that you care about.
I guess that it takes two different mindsets to in order to appreciate the pros and cons of dynamic vs static programming. There are certainly many pros for dynamic programming, but I'm more comfortable thinking about problems in generic terms where every relation and constraint is laid bare in front of me, one level removed from the actual implementation.
> The logic of type hint is not bad but sadly I think that type hint are making python source code messy and unreadable.
Compared to legacy Python, yes.
Compared to verbose language like Java, no. Python typing is equal or less verbose than Java (unless you use "var" in Java).
Python has union types, and you can type something as a container type with no type parameters.
Type hinting in python is a bit of a sticky plaster.
We have pyre enforcement at work, the problem is, that it has been gradually turned on over time, so some stuff is pyre compliant (or just strategically ignored) and some stuff isnt, so when you open some old code to do something, you have a million errors to deal with.
That would be fine if types were enforceable. in runtime type hinting does shit all.
I would dearly love a "strict" mode where duck typing is turned off and variable are statically typed. However I suspect that will never happen, even though it'd speed up a load of stuff if done correctly (type inference happens a lot)
I suspect to use type hints properly, I'd need to think a bit more C-like and create dataclasses as types to make things more readable, rather than using Dict[str,int] or what ever.
There are some other ways of expressing names for types, once you start using typing. There are typevars, enums and using the "|" to separate options, there are TypedDict, NamedTuple, Union, Literal, Optional, and probably more. Not everything needs to be a dataclass.
This is what we ended up using with mypy so we could add type checks to our CI without having to fix every single typing error:
https://github.com/orsinium-labs/mypy-baseline
The problem (in my opinion) is that Python gives you the tools (and perhaps even encourages you) to write code that would benefit from typing.
It's perfectly feasible to write maintainable, well-designed code in a dynamic language. I've worked with some extremely robust and ergonomic Clojure codebases before, for example. However, in Clojure, the language pushes you into its own "pit of success".
Personally, I never feel that with Python.
> yet another Python thing I'd have to try to keep in my mind despite it being months since I used them last.
Typing hints are also a moving target: they have changed, sometimes significantly, on every minor Python release since they came to be.
The `Optional` type came and went (being replaced by the new Union syntax.) Type classes are usually born in the `typing` module, then some of them get moved to `collections`, `abc`, or `collections.abc`. Some types have even been moved several times. `TypeAlias` came and went (replaced by `type`). `List` became `list`. `Tuple` became `tuple`. Forward declarations require a workaround by using string literals. But in Python 3.14, they no longer do, and the workaround will become deprecated.
I'm an evangelist for static type checking, and I never write a Python function signature without complete typing annotations. I also think that typing annotations in Python are evolving in a good way. However, for long-lived Python scripts, I highly recommend that developers are aware of, and factor in, the effort necessary to keep up with the evolution of static typing across Python versions. That effort spent on migrating is going to come on top of the mental load that typing hints already add to your plate.
> PPS: I think my ideal type hint situation would be if I could create distinct but otherwise unconstrained types for things like function arguments and function returns, have mypy or other typing tools complain when I mixed them
Cleaner/safer function args and return types is a common motivation for dataclass. has benefits over many/complex args besides typing too.
I typehint the stuff that is easy. My observations about typehinting in Python track the 80:20 rule or even a 90:10 rule. You get about 80% benefit for typhinting the easy 20%.
this is not a just a python problem but a problem in many dynamic languages.
switching to the typed variants whether typecript, python type-hints, mypy etc will force you to do the dance to make the compiler happy instead of working on code.
which is why for me - JSDoc is really good - it uses types for documentation and ends there.
Sometimes I feel like we need an analog to javascript/typescript. Ptypethon if you will.
Absolutely. The main problem with python typing is that checking types is optional. A dialect with mandatory types (with inference) and runtime/load-time checking would be great.
It’s kind of cool that type hints can be reflected on to do things (see Pydantic). Other than that I find it pretty cumbersome to use in practice, coming from TypeScript. Semi-relatedly I also dislike Python’s different ways to access objects/dicts, it feels arbitrary and cumbersome.
Program the way you like it and it should be fun.
If you are at work doing professional programming then typing helps avoid bugs and makes programming more of a reliable and robust process.
But doing your own thing, doing little utilities, banging out quick stuff, do exactly what makes you happy.
Programming should be fun not a chore and if types make it a chore for you then drop them.
Is that a double negative in your title ? Or is it an inside joke I didn't get ?
Ragebait title to get you to click it
You can't never have not enough of no good thing!
That's why they are not unspecified to be optional.