Note that the variables are safely and correctly interpolated at compile time. And it's type checked across the boundary too, by checking (at compile time) the column types with the live database.
Yes, what you did is strictly more powerful than what the Python people did. And you did it 20 years ago. Well done, have an upvote. And yet, here we are in 2025 with Python popularity growing unstoppably and (approximately) no one caring about OCaml (and all the other languages better than Python). It makes me sad.
But my two cents is that we're pretty lucky it's python that has taken off like a rocket. It's not my favorite language, but there are far worse that it could have been.
I'm switching between C, OCaml, Python, bash & Rust roughly equally every day (to a lesser extent, Perl as well). Not everything is what gets on the front page of HN.
It's interesting how the majority has explicitly chosen NOT to use the "better" languages. Is the majority really that bad in their judgment? Or is it that "better" is actually defined by adoption over time?
It's clearly better in their opinion, they just aren't optimizing for the same metrics that you are. Python is better because it's easy for people to learn, imo.
If someone is challenged figuring out a venv and they're not an absolute beginner, perhaps they aren't cut out to work in technology. There are countless subjects in the field more challenging and complicated to wrap one's brain around.
gp's claim is not "its easy to learn". It's not just the concept -- it's the ergonomics, absolutely terrible footguns (especially when dealing with global wheels that can screw up your running system), and the hidden state.
When would you want to interface between a specific project and the global Python environment running your system? If there's ever a time when "lock the project into a venv and don't cross-contaminate its dependencies with the global Python environment" isn't the answer, that sounds like a corner case. Let each project be its thing by itself.
On modern Linux you can type `python` at the command prompt and get a REPL. On Windows you download an installer from the official website (just like one usually does to install anything on Windows), then use `py` at the command prompt.
You don't need to `import` anything to start teaching Python. Even then you can do quite a lot with the standard library. Even then, unless you're using 3.11 or later on Linux you can let Pip install with `--user` until you actually need to isolate things between projects. (And even with new Python on Linux, the instructor can typically avert this by just installing a separate Python in `/usr/local/bin` for example. Yes, that's "cheating", depending on the classroom environment. But that's part of the point: installation hurdles are hurdles for self-learners, not for students.)
You only need to learn about virtual environments once you have projects with mutually conflicting dependencies, and/or once you're at a point where you're ready to publish your own software and should be learning proper testing and development practices. (Which will be largely orthogonal to programming, and not trivial, in any language.)
Teachers teach Python because it's easy to teach while still being relevant to the real world, in particular because boilerplate is minimized. You don't have to explain jargon-y keywords like "public" or "static" up front. You don't have to use classes for quite some time (if ever, really). You can express iteration naturally. Types are naturally thought of in terms of capabilities.
In my mind, Python has all the pedagogical advantages of Lisp, plus enough syntactic cues to prevent getting "lost in a sea of parentheses". (Of course, it lacks plenty of other nice Lisp-family features.)
> In my mind, Python has all the pedagogical advantages of Lisp, plus enough syntactic cues to prevent getting "lost in a sea of parentheses". (Of course, it lacks plenty of other nice Lisp-family features.)
> Peter Norvig here. I came to Python not because I thought it was a better/acceptable/pragmatic Lisp, but because it was better pseudocode. Several students claimed that they had a hard time mapping from the pseudocode in my AI textbook to the Lisp code that Russell and I had online. So I looked for the language that was most like our pseudocode, and found that Python was the best match. Then I had to teach myself enough Python to implement the examples from the textbook. I found that Python was very nice for certain types of small problems, and had the libraries I needed to integrate with lots of other stuff, at Google and elsewhere on the net.
Basically, that it's better pedagogically because it looks like pseudo-code and it's easy to get up and running quickly.
Which is valid, but frustrating to see it lead to actual adoption outside of pedagogy. That property is entirely orthogonal to, almost at odds with, what makes a good programming language for medium to large production quality applications.
If we used that logic elsewhere in life we’d all be playing the flute and cycling around on tricycles and balance bikes. But for some reason in tech it’s all about Hello World.
The story of the winner being scrapy market entrants that are lower-cost (...of learning, in the case of python) and good-enough-quality (...than OCaml, Lisps, Haskel, definitely not JS or Java) is not a new one. I don't subscribe to your analogies.
> You don't have to explain jargon-y keywords like "public" or "static" up front.
patently not true. you dont get too far into python -- especially if you are reading (or copypastaing) other People's code -- before you see if __name__ == "__main__" and any potential future programmer will rightfully ask "what the absolute fuck is this"
even "def" is kind of a weird fucking keyword.
Don't get me started about teaching beginners which datatypes are pass by reference and which are pass by value.
try explaining to an elementary school student why
> Don't get me started about teaching beginners which datatypes are pass by reference and which are pass by value.
If they are beginners to programming, you wouldn't teach them those terms in the context of Python, because neither of those terms map to Python argument passing; Python has one form of argument passing, and it doesn't map closely to the intuition that experienced programmers in languages that have pass by reference and pass by value have about those things. Likewise, the only thing you'd teach someone new to Python that is experienced in languages where those terms are useful is that that distinction is irrelevant in Python, which is pass by assignment (sometimes also called pass by object reference, but pass by assignment is a much more useful description IMO, because argument passing works exactly like assignment to a new variable name.)
> try explaining to an elementary school student why
def foo(a):
a = a + 1
> doesn't change the caller's variable but
def bar(a):
a.append(1)
> does.
But, that's easy, if you've first taught them how variables, assignment, and mutation work in Python without getting function calls in the way, because it is exactly the same as this
a = 1
b = a
b = a + 1
print(f"{a=}, {b=}")
vs.
a = [1]
b = a
b.append[1]
print(f"{a=}, {b=}")
Argument passing is just assignment to a new variable that exists in the scope of the function. Methods which mutate an object affect the object no matter what variable you access it from, assignment operations affect only the variable they assign to. That's exactly the same behavior in one scope as it is between the function and caller scopes.
And this distinction has nothing to do with data types, but only with the operations performed (the only connection to data types is that immutable types have no mutation operations in the first place.) You can tell its not about data types because you can use the same types as the second excerpt, and the operations of the first, and get the same results as the first (which shares operations) and not the second (which shares datatypes):
a = [1]
b = a
b = b + [1]
print(f"{a=}, {b=}")
If you understand how assignment and mutation works in one scope, you understand how argument passing works. Trying to teach a distinction that exists between how different operations affect variables that initially reference the same object as a distinction about how datatypes are passed as arguments is confusing, because you as a teacher are presenting the source of the difference in behavior as originating in a completely different place than where it actually comes from. That's not a problem with Python being complex, it is a problem with you taking a very simple thing and making it complex by ascribing it to a source that is completely irrelevant to what is actually going on.
> you dont get too far into python -- especially if you are reading (or copypastaing) other People's code -- before you see if __name__ == "__main__"
First off, if you are teaching someone, you are showing that person the code, and not allowing copy-and-paste.
Second, no, that comes up much less often than you'd expect.
Third, it's the same as `if value == 'example':`. Underscores are not jargon.
Fourth, it's trivially searchable. That's the part where you can copy and paste - into a search engine, which will immediately find you several competent explanations such as https://stackoverflow.com/questions/419163 .
> even "def" is kind of a weird fucking keyword.
Admittedly a poor choice, but not a deal breaker. You need the concept of functions to do programming. But you don't need the concept of data hiding, nor do you need any of the multiple, barely-related concepts described by the term "static".
> Don't get me started about teaching beginners which datatypes are pass by reference and which are pass by value.
There's nothing to explain. They are all pass by value, per the strict meaning of those terms.
Those terms have been widely seen as less than ideal for decades, however, because they fail to account for variables with reference semantics (i.e., what Python uses - which are sometimes called "names"). A more modern term is "pass by assignment", which correctly describes all variable passing in Python: passing an argument to a parameter is a form of assignment, and works the same way as assigning a value to a name.
This is far less complex than C#, in which user-defined types may have either value semantics or reference semantics, and which supports both pass by assignment and two separate, true forms of pass by reference (for initialization and for modifying an existing object). And certainly it's less complex than whatever C++ is doing (see e.g. https://langdev.stackexchange.com/questions/3798 ).
> try explaining to an elementary school student why
First: if someone gives you a bag with three apples in it, you can put another apple in the bag and give it back, and the other person will have a bag with four apples in it. But if you add 3 + 1, that doesn't change the meaning of 3. These are simple ideas that an elementary school student already understands.
Second: from extensive experience teaching beginners (never mind that you are moving the goalposts now), it makes no sense to get into the theory. It's not helpful. A student who can ask about this has already lost the plot, because the two examples use completely different syntax (a method call versus an assignment) so they shouldn't be expected to work similarly. You avoid this problem by using more precise language early on. "Change" is not an appropriate word here.
Third: you give this example because you think that `bar` (and you imply by your naming that a list is being passed) demonstrates pass by reference. This is simply incorrect. Please read https://nedbatchelder.com/text/names1.html.
Fourth: your use of profanity and the overall tone of your writing suggests that you simply don't like the fact that Python works the way that it does. This is not a good look IMO.
Just for the record, I've been in variations of this discussion countless times. I know what I'm talking about. All links above are in my bookmarks.
In my experience people have to first figure out what the hell numpy is and how to get it (venv, conda, pip, uv, uvx, …) because python arrays are shit, and so people fix that wart with an external C library. Then they notice that some other dependency requires a previous python version, but their python is installed globally and other dependencies were installed for that. These are uniquely python-specific problems. Lisp doesn’t have those problems
Did they try using a search engine? But more to the point, if they don't understand what it is, how did they find out it exists?
> how to get it (venv, conda, pip, uv, uvx, …)
uvx is a command from the same program as uv; venv is not a way to obtain packages; and the choice here isn't a real stumbling block.
> because python arrays are shit
I can't say I've seen many people complain about the standard library `array` module; indeed it doesn't seem like many people are aware it exists in the first place.
If you're talking about lists then they serve a completely different purpose. But your use of profanity suggests to me that you don't have any actual concrete criticism here.
> Then they notice that some other dependency requires a previous python version
Where did this other dependency come from in the first place? How did they get here from a starting point of dissatisfaction with the Python standard library?
> but their python is installed globally and other dependencies were installed for that.
... and that's where actual environment management comes in, yes. Sometimes you have to do that. But this has nothing to do with teaching Python. You have necessarily learned quite a bit by the time this is a real concern, and if you were taught properly then you can self-study everything else you need.
> These are uniquely python-specific problems.
No other languages ever require environment management?
You don’t need to teach it to a beginner. The first of learning doesn’t need more than the standard library and when you need more than that you’re either giving them the single command necessary to run or, more likely, having them use a template project where a tool like Poetry is doing that automatically.
What this usually hits isn’t that managing Python packages is hard in 2025 but that many people do not learn how their operating system works conceptually until the first time they learn to program and it’s easy to conflate that with the first language you learn.
Let me introduce you to uv[0]. And yes it does say something that this tool isn't written in Python, but I'd say there's even more to be said that so many are trying to support Python.
yeah the reason why that's not an answer is because another half of users will say use poetry. if you want to do bioinformatics, people will insist on conda. then your team will say "use rye" and these strategies are somewhat compatible but ultimately mutually incompatible in ways that will drive you mad every time you hit a tiny snag that nonetheless grinds your attempt to just fucking run code to a halt.
It has become successful largely because it has always had really good foreign function interface. If you have a scientific or mathematical library laying around in C, then you could wire it up to Python, and then suddenly you have all the flexibility of a (fairly clean) scripting language to orchestrate your high speed C.
someone learning python as their first language knows so little its perfectly fine to let them pollute their global environment. Someone who knows other languages can understand what venv is for.
Instead they can type python to open a shell and use python to immediately run their file.
It's not, because you can fuck up system/unrelated app python dependencies and in extreme cases have to reinstall OS. Thankfully as developers migrate away from python/adopt strategies like flatpak this is less of a problem.
And yet that momentum and ecosystem wouldn't have been achieved in the first place if there weren't enough merits in the language to trigger and maintain that interest.
Some languages definitely had people gravitate towards them due to being innovative in a given space, but in many of those cases, the comparative advantage was lost to other languages/techs/frameworks that simply failed to gain a market share "equal to their innovative contribution" due to the first comer's advantage.
I think you're being too unfair. People aren't dumb.
It's also about how much better.
Beyond a decent enough type system, the advantages start to flatten and other factors start to matter more.
Can't speak too much for python, but as someone who's written large amounts of code in OCaml and Typescript, the strictest compiler options for Typescript are good enough.
No, people aren't dumb. They're practical. And so they choose to do what is practical, which in this case is to choose Python. And, to me, that makes it the better language.
Aren't there other benefits to server-side parameter binding besides just SQL-injection safety? For instance, using PG's extended protocol (binary) instead of just raw SQL strings. Caching parameterized prepared statements, etc.
Also:
db.execute(t"QUERY WHERE name = {name}")
Is dangerously close to:
db.execute(f"QUERY WHERE name = {name}")
A single character difference and now you've just made yourself trivially injectible.
I don't think this new format specifier is in any way applicable to SQL queries.
Templates are a very different duck type from strings and intentionally don't support __str__(). The SQL tool can provide a `safe_execute(Template)` that throws if passed a string and not a Template. You can imagine future libraries that only support Template and drop all functions that accept strings as truly safe query libraries.
> Caching parameterized prepared statements, etc.
Templates give you all the data you need to also build things like cacheable parameterized prepared statements. For DB engines that support named parameters you can even get the interpolation expression to auto-name parameters (get the string "name" from your example as the name of the variable filling the slot) for additional debugging/sometimes caching benefits.
You solve that with an execute(stmt) function that requires you to pass in a template.
In Javascript, sql`where id = ${id}` is dangerously close to normal string interpolation `where id = ${id}`, and db libs that offer a sql tag have query(stmt) fns that reject strings.
> A single character difference and now you've just made yourself trivially injectible.
No; a single character difference and now you get a `TypeError`, which hopefully the library has made more informative by predicting this common misuse pattern.
> Aren't there other benefits to server-side parameter binding besides just SQL-injection safety? For instance, using PG's extended protocol (binary) instead of just raw SQL strings. Caching parameterized prepared statements, etc.
All of which can be implemented on top of template strings.
> A single character difference and now you've just made yourself trivially injectible.
It's not just a one character difference, it's a different type. So `db.execute` can reject strings both statically and dynamically.
> I don't think
Definitely true.
> this new format specifier is in any way applicable to SQL queries.
The first mistake we're going to see a library developer make is:
def execute(query: Union[str, Template]):
Maybe because they want their execute function to be backwards compatible, or just because they really do want to allow either raw strings are a template string.
> they really do want to allow either raw strings are a template string.
I’d consider that an invalid use case:
1. You can create a template string without placeholders.
2. Even if the caller does need to pass in a string (because they’re executing from a file, or t-strings don’t support e.g. facetting) then they can just… wrap the string in a template explicitly.
I didn’t explicitly mention this in my post but, yes, the Template type is designed with caching in mind. In particular, the .strings tuple is likely to be useful as a cache key in many cases.
This is true of many other things, which is why we have type checkers and linters to be perfectly rigorous rather than expecting humans to never make mistakes.
Click "parent" a few times and look at the code example that started this thread. It's using the same function in a way that can't distinguish whether the user intentionally used a string (including an f-string) and a t-string.
Yes, and the parent is misguided. As was pointed out in multiple replies, the library can distinguish whether an ordinary string or a t-string is passed because the t-string is not a string instance, but instead creates a separate library type. A user who mistakenly uses an f prefix instead of a t prefix will, with a properly designed library, encounter a `TypeError` at runtime (or a warning earlier, given type annotations and a checker), not SQL injection.
In this particular instance it can't, because there are 3 ways in question here, and it can't distinguish between correct intentional usage and accidental usage of an f-string instead of a t-string:
db.execute("SELECT foo FROM bar;")
db.execute(f"SELECT foo FROM bar WHERE id = {foo_id};")
db.execute(t"SELECT foo FROM bar WHERE id = {foo_id};")
The first and second look identical to execute() because all it sees is a string. But the second one is wrong, a hard-to-see typo of the third.
If f-strings didn't exist there'd be no issue because it could distinguish by type as you say. But we have an incorrect SQL-injection-prone usage here that can't be distinguished by type from the correct plain string usage.
My (and their) point is that's the already existing API. You're proposing a big breaking change, with how many frameworks and tutorials are built on top of that.
It's not like this is the first time APIs have been improved. There are many tools (e.g. deprecation warnings & hints in editors, linter rules) that can help bridge the gap - even if t-strings are only used for new or refactored code, it's still a big improvement!
There's also simply no hard requirement to overload an `execute` function. We have options beyond "no templates at all" and "execute takes templates and strings", for example by introducing a separate function. Why does perfect have to be the enemy of good here?
Because they're both passed to "execute", which can't tell between the f-string and a non-interpolated query, so it just has to trust you did the right thing. Typoing the "t" as an "f" introduces SQL injection that's hard to spot.
Assuming `execute` takes both. You could have `execute(template)` and `execute_interpolated(str, ...args)` but yeah if it takes both you'll have challenges discouraging plain-text interpolation.
It would have to be the other way around or be a (possibly major) breaking change. Just execute() with strings is already standard python that all the frameworks build on top of, not to mention tutorials:
> It would have to be the other way around or be a (possibly major) breaking change.
If it is going to reject the currently-accepted unsafe usage, its going to be a major breaking change in any case, so I don't see the problem. I mean, if you are lamenting it can't reject the currently-accepted SQL-interpolated-via-f-string because it can't distinguish it by type from plain strings with no interpolation, you are already saying that you want a major breaking change but are upset because the particular implementation you want is not possible. So you can't turn around and dismiss an alternative solution because it would be a major breaking change, that's what was asked for!
With t-strings you could run proper escaping over the contents of `some_file` before passing it to a shell.
I'd have to take a look at the order things happen in shell, but you might even be able to increase security/foot-gun-potential a little bit here by turning this into something like `stat "$( base64 -d [base64 encoded content of some_file] )"`.
Oh! I missed this one because I've been looking specifically at the Packaging forum rather than the PEPs forum. This looks like a brilliant use case. (I'm aiming for wide compatibility - back to 3.6 - with my current projects, but I look forward to trying this out if and when it's accepted and implemented.)
Now if only the overall `subprocess` interface weren't so complex....
The key point is that t-strings are not strings. Db.execute(t”…”) would throw an exception, because t”…” is not a string and cannot be interpreted as one.
In order for a library to accept t-strings, they need to make a new function. Or else change the behavior and method signature of an old function, which I guess they could do but any sanely designed library doesn’t do.
Handling t-strings will require new functions to be added to libraries.
> The problem is writing f instead of t => and this will silently work I assume (not a Python dev just trying to understand the language design)
In the fullness of time it has no reason to. Even in the worst case scenario where you have to compose the query dynamically in a way t-strings can’t support, you can just instantiate a Template object explicitely.
>yes but the bug is writing f instead of t and I assume f will just work
but it will not. f-strings and t-strings are not compatible types, they will not "just work". not unless somebody changes a library to make it just work. as long as nobody does that, it's not an issue.
You add a new API that takes templates only leaving the existing API in place. You (some releases later) deprecate the string API. You (some releases later, with clear advance warning of when it is coming) actually remove the deprecated API. "It's a big breaking change around a brand new feature", yeah, so you don't break anything around a brand new feature, it's not like this kind of transition is a new concept.
The execute function can recognize it as a t-string and prevent SQL injection if the name is coming from user input. f-strings immediately evaluate to a string, whereas t-strings evaluate to a template object which requires further processing to turn it into a string.
Then the useful part is the extra execute function you have to write (it's not just a substitute like in the comment) and an extra function can confirm the safety of a value going into a f-string just as well.
I get the general case, but even then it seems like an implicit anti-pattern over doing db.execute(f"QUERY WHERE name = {safe(name)}")
Problem with that example is where do you get `safe`? Passing a template into `db.execute` lets the `db` instance handle safety specifically for the backend it's connected to. Otherwise, you'd need to create a `safe` function with a db connection to properly sanitize a string.
And further, if `safe` just returns a string, you still lose out on the ability for `db.execute` to pass the parameter a different way -- you've lost the information that a variable is being interpolated into the string.
db.safe same as the new db.execute with safety checks in it you create for the t-string but yes I can see some benefits (though I'm still not a fan for my own codebases so far) with using the values further or more complex cases than this.
Yeah but it would have to be something like `db.safe("SELECT * FROM table WHERE id = {}", row_id)` instead of `db.execute(t"SELECT * FROM table WHERE id = {row_id}")`.
This is just extra boilerplate though, for what purpose?.
I think one thing you might be missing is that in the t-string version, `db.execute` is not taking a string; a t-string resolves to an object of a particular type. So it is doing your `db.safe` operation, but automatically.
Of course you can write code like that. This is about making it easier not to accidentally cause code injection by forgetting the call of safe(). JavaScript had the same feature and some SQL libraries allow only the passing of template strings, not normal strings, so you can't generate a string with code injection. If you have to dynamically generate queries they allow that a parameter is another template string and then those are merged correctly. It's about reducing the likelihood of making mistakes with fewer key strokes. We could all just write untyped assembly instead and could do it safely by paying really good attention.
agreed. but then you're breaking the existing `db.execute(str)`. if you don't do that, and instead add `db.safe_execute(tpl: Template)`, then you're back to the risk that a user can forget to call the safe function.
also, you're trusting that the library implementer raises a runtime exception if a string a passed where a template is expected. it's not enough to rely on type-checks/linting. and there is probably going to be a temptation to accept `db.execute(sql: Union[str, Template])` because this is non-breaking, and sql without params doesn't need to be templated - so it's breaking some stuff that doesn't need to be broken.
i'm not saying templates aren't a good step forward, just that they're also susceptible to the same problems we have now if not used correctly.
Yeah, you could. I'm just saying that by doing this you're breaking `db.execute` by not allowing it to take it string like it does now. Libraries may not want to add a breaking change for this.
What does db.safe do though? How does it know what is the safe way of escaping at that point of the SQL? It will have no idea whether it’s going inside a string, if it’s in a field name position, denotes a value or a table name.
To illustrate the question further, consider a similar html.safe: f"<a href={html.safe(url)}>{html.safe(desc)</a>" - the two calls to html.safe require completely different escaping, how does it know which to apply?
Some SQL engines support accepting parameters separately so that values get bound to the query once the abstract syntax tree is already built, which is way safer than string escapes shenanigans.
I’d always prefer to use a prepared statement if I can, but sadly that’s also less feasible in the fancy new serverless execution environments where the DB adapter often can’t support them.
For me it just makes it easier to identify as safe, because it might not be obvious at a glance that an interpolated template string is properly sanitised.
> and an extra function can confirm the safety of a value going into a f-string just as well.
Yes, you could require consumers to explicitly sanitize each parameter before it goes into the f-string, or, because it has the structure of what is fixed and what is parameters, it can do all of that for all parameters when it gets a t-string.
The latter is far more reliable, and you can't do it with an f-string because an f-string after creation is just a static string with no information about construction.
> Then the useful part is the extra execute function you have to write
Well, no, the library author writes it. And the library author also gets to detect whether you pass a Template instance as expected, or (erroneously) a string created by whatever formatting method you choose. Having to use `safe(name)` within the f-string loses type information, and risks a greater variety of errors.
Your "old" db.execute (which presumably accepts a regular old string) would not accept a t-string, because it's not a string. In the original example, it's a new db.execute.
Using a t-string in a db.execute which is not compatible with t-strings will result in an error.
Using a t-string in a db-execute which is, should be as safe as using external parameters. And using a non-t-string in that context should (eventually) be rejected.
Yes, but if a function accepts a template (which is a different type of object from a string!), either it is doing sanitization, or it explicitly implemented template support without doing sanitization—hard to do by accident!
The key point here is that a "t-string" isn't a string at all, it's a new kind of literal that's reusing string syntax to create Template objects. That's what makes this new feature fundamentally different from f-strings. Since it's a new type of object, libraries that accept strings will either have to handle it explicitly or raise a TypeError at runtime.
I'm not sure why you think it's harder to use them without sanitization - there is nothing inherent about checking the value in it, it's just a nice use.
You might have implemented the t-string to save the value or log it better or something and not even have thought to check or escape anything and definitely not everything (just how people forget to do that elsewhere).
I really think you're misunderstanding the feature. If a method has a signature like:
class DB:
def execute(query: Template):
...
It would be weird for the implementation to just concatenate everything in the template together into a string without doing any processing of the template parameters. If you wanted an unprocessed string, you would just have the parameter be a string.
I'm not. Again, you might be processing the variable for logging or saving or passing elsewhere as well or many other reasons unrelated to sanitization.
Taking a Template parameter into a database library's `execute` method is a big bright billboard level hint that the method is going to process the template parameters with the intent to make the query safe. The documentation will also describe the behavior.
You're right that the authors of such libraries could choose to do something different with the template parameter. But none of them will, for normal interface design reasons.
A library author could also write an implementation of a `plus` function on a numerical type that takes another numerical type, and return a string with the two numbers concatenated, rather than adding them together.
But nobody will do that, because libraries with extremely surprising behavior like that won't get used by anybody, and library authors don't want to write useless libraries. This is the same.
It's true that in theory `db.execute` could ignore semantics and concatenate together the template and variables to make a string without doing any sanitisation, but isn't the same true of the syntax it was claimed to replace?
Just because templates (or the previous syntax of passing in variables separately) could be used in a way that's equivalent safety-wise to an f-string by a poorly designed library does not mean that they add nothing over an f-string in general - they move the interpolation into db.execute where it can do its own sanitization and, realistically, sqlite3 and other libraries explicitly updated to take these will use it to do proper sanitization.
or other ways of manually interpolating the string - because `db.execute` can flag a `TypeError` if given a string (no matter how it was constructed) rather than a `Template` instance.
Python is not the first one to get this feature. It's been present in JS for some time now, and before that in C# (not sure if that's the origin or they also borrowed it from somewhere). Python adopted it based in part on successful experience in those other languages.
Assuming you also need to format non-values in the SQL (e.g. column names), how does the `execute` function is supposed to make the difference between stuff that should be formatted in the string vs a parametrized value?
However I think using the format spec that way would be dubious and risky, because it makes the sink responsible for whitelisting values, and that means any processing between the source and sink becomes a major risk. It's the same issue as HTML templates providing `raw` output, now you have to know to audit any modification to the upstream values which end there, which is a lot harder to do than when "raw markup" values are reified.
> rather than with some "out-of-band" logic.
It's the opposite, moving it to the format spec is out of band because it's not attached to values, it just says "whatever value is here is safe", which is generally not true.
Unless you use the format spec as a way to signal that a term should use identifier escaping rules rather than value escaping rules (something only the sink knows), and an `Identifier` wrapper remains a way to bypass that.
> Unless you use the format spec as a way to signal that a term should use identifier escaping rules rather than value escaping rules (something only the sink knows)
This should be quiet common in the SQL applications. It will be nice to write t"select {name:id} from {table:id} where age={age}" and be confident that the SQL will be formatted correctly, with interpolations defaulting to (safe) literal values.
The article does mention that the function receiving the template has access to those formatting options for each interpolation, so presumably you could abuse the ones that are available for that purpose?
I think it's just giving an error because a valid AST can't be made, which means valid bytecode can't be made. "<word> <word>" is only valid syntax if one is a reserved word. `nonlocal(foo)` is just fine, of course.
No, it gives an error because `nonlocal foo` requests that the name `foo` be looked up in a closure, but `f` doesn't have such a closure (the `foo` defined outside the function is global instead). `nonlocal` is the same sort of keyword as `global` but for enclosing functions instead of the global namespace; see also https://stackoverflow.com/questions/1261875 .
Here's the statement checking code, which I believe is pre-AST [1]. I would have to dig more to see if that check is there to prevent invalid AST or to just "help the user" (would depend on how they reference the original variable I suppose).
But wow, that's the first time I've seen "nonlocal". In the ~100 packages I have installed, I see 0 usages!
Well, yes, not a lot of people write closures except perhaps when they implement decorators. So there's ordinarily no non-local scope to worry about. People tend to write classes instead, because that's what's familiar.
I think this would have solved the log4j vulnerability, no?
As I understand it, log4j allowed malicious ${} expansion in any string passed to logging functions. So logging user generated code at all would be a security hole.
But Python's t-strings purposely _do not_ expand user code, they only expand the string literal.
Yes and your example is the hero case because it isn't just sugar. A t-string implementation for SQL will of course escape the values which is a common security issue.
Now instead of being explicit all it takes is someone unfamiliar with t strings (which will be almost everyone - still few know about f strings and their formatting capabilities) to use an f instead and you are in for a bad time.
Any sane library will just error when you pass a string to a function that expects a template though. And that library will have types too so your IDE tells you before you get that far.
Such library functions tend to also accept a string as a valid input. E.g. db.execute from the GP usually works with strings to allow non-parametrized SQL queries.
> Such library functions tend to also accept a string as a valid input.
Also? They tend only to accept a string (possibly with some additional arguments, if there is an in-library way to handle parameterization) as input, because Template literally hasn't been an option. New APIs designed with Template available will look different.
A library can extend an existing database library like 'pg' so that PgClient#query() and PgPool#query() require string template statements.
That way 'pg' can continue working with strings, and people who want nice templated strings can use the small extension library, and the small extension library makes it impossible to accidentally pass strings into the query functions.
That is an issue, but essentially it boils down to the existing risk of unknowledgeable people not escaping untrusted inputs. The solution should be more education and better tooling (linters, SAST), and t-strings are likely to help with both.
I suppose lack of overlap in the "interface surface" (attributes, including callables) between `str` and `Template` should nip the kind of issue in the bud -- being passed a `Template` and needing to actually "instantiate" it -- accessing `strings` and `values` attributes on the passed object, will likely fail at runtime when attempted on a string someone passed instead (e.g. confusing a `t`-string with an `f`-string)?
> In addition, I hope that the tooling ecosystem will adapt to support t-strings. For instance, I’d love to see black and ruff format t-string contents, and vscode color those contents, if they’re a common type like HTML or SQL.
This is such a strange take on t-strings. The only way for anything to infer that the template string is supposed to turn into valid HTML or SQL is to base it of the apparent syntax in the string, which can only be done in an ad-hoc fashion and has nothing to do with the template string feature.
The way the feature has been designed there is no indication in the string itself what type of content it is or what it will eventually be converted to. It’s all handled by the converting function.
As others have added, something like sql”select * from {table}” would have been able to do this, but there’s not even any guarantees that something that is in a template that will be converted into valid sql by a converting function should be any type of valid sql prior to that conversion. For all you know t“give me {table} but only {columns}” might be a converted into valid sql after the template is processed.
As Paul mentioned, we spent quite a lot of time considering these issues as PEP 750 came together. In the end, we concluded (a) the PEP leaves open quite a few potential approaches for tools to adopt (not just the one you suggest, as others here have pointed out), and (b) it's ultimately something that the broader tooling community needs to rally around and should probably be out of scope for the PEP itself.
So, with that background in mind, I am indeed hopeful we'll see the ecosystem adapt! :-)
I get that. But that “ecosystem might adapt” has absolutely nothing to do with t-strings is what I’m saying. Anyone who wants to validate sql will have to just look through strings guess that it’s sql and validate and highlight it. This could be done before and after just the same. The template strings by nature of just being “strings that will turn into something else at some later point” do nothing to further this in any way.
And highlighters and static analyzers will key off of this.
JavaScript's tagged template literals are actually about as flexible as this, since you can dynamically choose the tag function, it's just very rare to do so, so tools assume a lot based on the name of the function. Python tools can basically do the same thing, and just not support t-strings that aren't nested inside a well-named processing function.
> And highlighters and static analyzers will key off of this.
Then the t is redundant and we don’t need t strings involved in any way. This would work just as well. In fact better by using html("<h1>Hello</h1>") and have it return a html object instead of returning a just a string which is what t-strings will do. So literally the only contribution from t-strings in that context is to make things worse.
If they had just standardized some way of putting semantics into the template everything would be better. Let us define a “sql = temlate(….)” and use sql”…”
The original PEP and the original discussion had this in scope. We removed it to let this emerge later. There are different ways to signal the language -- some more friendly to tooling, some more robust.
PyCharm (as well as the other JetBrains IDEs) has for at least 10 years supported language injection in Python strings using a comment [0]. Worst case scenario there's no reason whatsoever that that couldn't be used by formatters to apply auto-formatting in a structured and deterministic way.
But that's far from the only option, either! IntelliJ for Java also supports annotations on arguments [1] that then mean that you get syntax highlighting everywhere you use a string literal with the given function:
public void query(@Language("SQL") String sql)
In Python, typing.Annotated appears to have been specifically designed for purposes like this [2]:
> If a library or tool encounters an annotation Annotated[T, x] and has no special logic for the metadata, it should ignore the metadata and simply treat the annotation as T. As such, Annotated can be useful for code that wants to use annotations for purposes outside Python’s static typing system.
So something like this should be perfectly viable:
SQL = Annotated[Template, "language", "SQL"]
def query(sql_query: SQL):
# do stuff with Template to sanitize
query(t"SELECT * FROM foo WHERE bar={bar}")
Now, where you're right is that we shouldn't actually require template strings to accomplish this! As noted, JetBrains has been doing this since forever. But maybe template strings will be useful enough for purposes like this that the tooling will actually evolve to support it in formatters and other editors (and maybe PyCharm can get some of the better support that Java has from JetBrains).
Couldn’t you do this with a type annotation? e.g. SQLAlchemy could have a SQL type so tools like mypy could see a Template instance and confirm you’re using it safely but Black, Ruff, or SQLFluff could look for the more specialized Annotated[Template, SQL] to realize that the template could be formatted as SQL, and something like Django could even have Annotated[Template, Email], Annotated[Template, HTML], or Annotated[Template, JSX] to indicate what context the same templating syntax is targeting.
This is what we discussed in the first revision of the PEP (the use of `Annotated`.) But we found out: linters don't know anything about the Python type system.
We hope to get a community around all of this, stuff at PyCon US, EuroPython, etc. and work some of this out. The JSX/TSX world really has good tooling. We can provide that for those that want it, perhaps better on some aspects.
Interesting, thanks for the background. I’ve been curious what Astral is going to do in the space but also worry about what happens when their funding runs out.
> The only way for anything to infer that the template string is supposed to turn into valid HTML or SQL is to base it of the apparent syntax in the string
Not the only thing. You can also look at how it is used. Your editor could know of how some popular libraries use t-strings, track which t-strings get passed into functions from those libraries, and use that to assume what grammar the t-string should follow.
Is that cheating? In some sense, yes, but it also is useful and likely will be worth it for quite a few programmers.
Will this allow neat SQL syntax like the following?
city = 'London'
min_age = 21
# Find all users in London who are 21 or older:
users = db.get(t'
SELECT * FROM users
WHERE city={city} AND age>{min_age}
')
If the db.get() function accepts a template, it should, right?
This would be the nicest way to use SQL I have seen yet.
You would need triple-quotes to span multiple lines, but yes, that is exactly how it's intended to work. `db.get` will receive a Template instance, which stores string parts something like `('\n SELECT * FROM users\n WHERE city=', ' AND age>', '\n')` and interpolated parts like `(Interpolation('London'), Interpolation(21))`. It's then responsible for assembling and executing the query from that.
That’s the sort of thing people have built with the equivalent feature in JavaScript, so it should do. Eg https://github.com/andywer/squid is a nice example.
Isn't the actually proper way to use prepared statements anyway? If we are doing that properly, then what does this t string business buy us for SQL usage from Python?
Except to maintain backwards compatibility we're probably going to get new methods that only accept templates, completely circumventing any effort to stop people passing in strings.
Prepared statements were the recommended way to run SQL queries when I was starting with PHP 15 years ago, anyone writing code vulnerable to SQL injection at this point should not be writing code.
Thanks, I hate it. While it's nice syntactic sugar, the difference between an SQL injection vulnerability and a properly parametrized query is now a single letter that's easily missed
The t-string produces a Template object without a __str__() method. You can’t mistakenly use an f-string in its place. Either the code expects a string, in which case passing it a Template would blow it up, or the code expects a Template, in which case passing it a string would blow it up.
> or the code expects a Template, in which case passing it a string would blow it up.
That's where the problem is though -- in most cases it probably won't blow up.
Plenty of SQL queries don't have any parameters at all. You're just getting the number of rows in a table or something. A raw string is perfectly fine.
Will sqlite3 really disallow strings? Will it force you to use templates, even when the template doesn't contain any parameters?
You can argue it should, but that's not being very friendly with inputs, and will break backwards compatibility. Maybe if there's a flag you can set in the module to enable that strict behavior though, with the idea that in a decade it will become the default?
It's one extra letter to "force" for an unparameterized query over a "raw string". The t-string itself works just fine without parameters.
There's definitely a backwards compatibility hurdle of switching to a template-only API, but a template-only API doesn't look that much "less friendly" with inputs, when the only difference is a `t` before every string, regardless of number of parameters.
Sure, but it's just I don't have to do that anywhere else.
I never put an f in front of a string if I'm not putting variables within it.
And I'm generally used to Python inputs being liberal. I can usually pass a list if it expects a tuple; I can pass an int if it expects a float; often I can pass an item directly instead of a tuple with a single item. Regex functions take regular strings or regex strings, they don't force regex strings.
Being forced to use a single specific type of string in all cases is just very different from how Python has traditionally operated.
It's safer, I get that. But it's definitely less friendly, so I'll be curious to see how module maintainers decide to handle this.
> Being forced to use a single specific type of string in all cases is just very different from how Python has traditionally operated.
Maybe that's partly the disconnect here? "t-string" is probably a confusing colloquial name because they aren't strings, they are Templates. The runtime type is a Template. It is a very different duck-type from a string. As a duck-typable object it doesn't even implicitly or explicitly act like a string, there's intentionally no `__str__()` method and `str(someTemplate)` doesn't work like you'd expect. It shouldn't be a surprise that there is also no implicit conversion from a string and you have to use its own literal syntax: it isn't a string type, it's a Template type.
Python here is still liberal with respect to Templates (it is still a duck type). If a function expects a Template and you don't want to use the t"" shorthand syntax nor use the Template constructor in string.templatelib, you just need a simple class of object that has an `__iter__()` of the correct shape and/or has `strings` and `values` tuples.
Sure, it may make sense for some types of APIs to support a Union of str and Template as "liberal" options, but it's a different class of liberal support from Union of list and tuple or Union of int and float which are closer "domains" of types. A Template isn't a string and at runtime looks nothing like one (despite how syntactically it looks like one at "compile time"). Given `__iter__()` in Template, it may make more sense/would be more "natural" to Union Template with List or Tuple more than with a single string.
Er… that’s just not correct? Python can be more liberal but it’s not always. It depends entirely on the tooling. Libraries will take time to catch up but I can definitely see people creating libraries that enforce t-strings, even if they’re deconstructing them under the hood for legacy libraries.
It works fine. It doesn't work with a float, which is for good reason. That's the whole point -- its general philosophy is to be pretty liberal with types, except when there's a good reason not to be. Heck, you can even do dumb things like:
4 + True
And get 5 back. If that's not "blindly interchanging different types for no good reason" then I don't know what is. You can even multiply your Decimal object by False and get an answer...
Or it's like my original example -- the Regex module isn't restricted to r-strings. It happily works with regular strings. Python's general philosophy is to handle input liberally. Even type hinting is an add-on. Now, it doesn't go as far as JavaScript in allowing e.g. "4"+1, but it's still awfully liberal. I just don't see how you can claim otherwise.
I know about Decimal/int mixing but that’s for a good reason: it’s fine to intermix here. But not for floats (precision issues). The bool/int mixing isn’t “good.” It’s a bad implementation detail that Python is stuck maintaining forever. I’m actually stunned that you think to use this as an example when I think I’d fire any programmer that did that in my team for gross negligence.
The reason it works is because Python functionally has no bool type. True and False are just integers with names. It’s stupid and shouldn’t work like that but it does for historic reasons.
Your example of regex makes no sense either. There is no difference between strings and r-strings. They’re literally the same thing to the interpreter, so how could the regex functions enforce you use r-strings? Maybe they should be different but, for historic reasons, they can’t be without Python 4.0.
> The reason it works is because Python functionally has no bool type. True and False are just integers with names.
This has not been true since around 2.4 or 2.5. The oldest Python I have available to me currently is 2.7, and this holds then, as it does now in 3.13:
Prior to having a bool type, Python didn't even have True/False keywords.
The reason something silly like `4 + True` works is because the bool type implements `tp_as_number` [0]. The reason it works this way is intentional because it would been a Python 3 str-style debacle if ints and bools were not interchangeable.
> I’m actually stunned that you think to use this as an example when I think I’d fire any programmer that did that in my team for gross negligence.
You seem to be having a different conversation than I am.
I'm just describing Python as it is. I'm not defending it. I know why you can add True to a number, or else I wouldn't have come up with the example. And I know perfectly well that r-strings are just strings. Python easily could have made them a distinct object, to force people from ever making backslash errors, and restricted Regex functions to them, but didn't.
My only point has been, "Pythonic" things tend to be pretty liberal in what they accept. Type hints aren't even enforced, when they exist at all. You seem to think it shouldn't be that way. Great! But regardless, claiming it's not that way -- that Python is somehow this strict language -- is just mischaracterizing it.
> My only point has been, "Pythonic" things tend to be pretty liberal in what they accept
Being able to use a string as a string and an int as an int are not “pretty liberal in what they accept,” it’s just programming language theory 101! I think you’re mistaking duck typing for “liberal acceptance,” which are not the same thing. There’s always been an expectation that you should use compatible interfaces, even within the standard library. I’ve been bitten enough times by passing a generator in when a function expects a list, for example.
I'm not mistaking it at all. Yes, duck typing is very much liberal acceptance, but Python code tends to go much farther. I could give a million examples -- like how in Python's isinstance() the second argument can be a type or a tuple of types, or in sqlite3 that you can run queries on a connection or on a cursor, and don't even get me started on Matplotlib or Numpy. It's just idiomatic in Python to make things easy for the programmer by accepting multiple types when possible. If you don't recognize this as the common Python pattern, I literally don't know what else to tell you.
For the reasons discussed above, I'm not sure that it will be the case for t-strings. I think it'll take a little while for frameworks/libraries to adapt (while still maintaining backward compatibility) and a while for best practices to find their way into our linting and other tools.
If you can use a string anywhere you can use a t-string, then a non parametrized t-string is a code smell (lining error). If there is a dedicated template-string API, then there is the implicit threat you are breaking backwards compatibility to stop using regular strings.
A library writer ultimately has to decide if they accept both types. For a database cursor, do you take regular strings + parameter arguments and template strings? Or dedicate a new API to the idea?
cursor.execute(“select * from x where foo=?”, {foo=1})
# while also allowing
cursor.execute(t“select * from x where foo={foo}”)
#Vs
cursor.executetemplate(“select * from x where foo={foo}”)
If ‘execute’ takes string and t-string, then I would consider it a problem to use a t-string without parameters. If there is a novel API just for t-strings, then you are implying widespread breaking changes as you have a schism between the two ways of providing parameters.
My point was that library authors will need to consider this carefully. If you're writing a library where injection attacks matter, then -- long term -- you almost certainly do not want a single method that accepts `Union[str, Template]`. You probably either want to avoid accepting `str` entirely, or perhaps provide two separate methods. Some period of deprecation seems inevitable.
I think it's way more likely that existing libraries will introduce new methods that use t-strings and are type safe, rather than entirely defeat the purpose of having a t-string API.
I'm guessing no existing functions will be extended to allow t-strings for this very reason. Instead, new functions that only accept t-strings will be created.
There's an obvious risk here, same as with strcpy (no, strncpy.. no, strlcpy... no, strcpy_s) that documentation tends to outlive code, and people keep pasting from tutorails and older code so much that the newer alternatives have a hard time cutting through the noise.
I would argue that as bad as some w3schools tutorials were, and copying from bad Stackoverflow answers, going back to MSA and the free cgi archives of the 90s, the tendency of code snippets to live on forever will only be excarbated by AI-style coding agents.
On the other hand, deprecating existing methods is what languages do to die. And for good reason. I don't think there's an easy answer here. But language is also culture, and shared beliefs about code quality can be a middle route between respecting legacy and building new. If static checking is as easy as a directive such as "use strict" and the idea that checking is good spreads, then consesus can slowly evolve while working code keeps working.
Do the python type checkers / linters / whatever have the ability to warn or error on calling certain functions? That would be nice to eventually enforce migration over to the newer functions that only take a t-string template
Yeah. A while back I was poking through some unfamiliar code and noticed that my editor was rendering a use of `datetime.utcnow()` as struck through. When I hovered it with my mouse, I got a message that that function had been deprecated.
Turns out my editor (vscode) and typechecker (pyright) saw that `datetime.utcnow()` was marked as deprecated (I know one can use the `@deprecated` decorator from Python 3.13 or `__future__` to do this; I think it was done another way in this particular case) and therefore rendered it as struck through.
And it taught me A) that `utcnow()` is deprecated and B) how to mark bits of our internal codebase as deprecated and nudge our developers to use the new, better versions if possible.
Can you do it for functions defined by other people, or only for functions that you defined?
I'm thinking in the general case, but motivated by this example of a 3rd party function that accepts a SQL query as a string, and we'd like everywhere in our codebase to stop using that and instead use the 3rd party function that accepts the query as a t-string
If it's not a completely new library written exclusively around templates, such code currently accepts strings and will most likely continue to accept strings for backwards compatibility.
In that case I don’t understand the security regression that t-strings cause (see GP). Before it was all just strings, but you had to make sure to use them in the correct place. Now you can still just use strings for backwards compat. but you can also move on to a distinctly-typed solution for SQL and the like.
That would be a great argument if Python wasn't a language that let you reach into the internals and define __str__() for things you shouldn't be defining it for. And that is something people will definitely do because, you know, they just need something to friggin work so they can get whatever ticket closed and keep some metric happy tracking time-to-close
I guess that is a misunderstanding on your side, about how templates work. Less hate and more love might help to avoid this type of hotheaded misconception ;-)
Why do you think changing a letter would cause a vulnerability? Which letter do you mean?
No sane library is going to do that. If they do let you pass a raw string it should be a different function with the risks clearly documented.
The thing this replaces is every library having their own bespoke API to create a prepared statement on their default/safe path. Now they can just take a template.
How about every library that wants to preserve backwards compatibility?
Or are you suggesting that e.g. every database module needs to implement a new set of query functions with new names that supports templates? Which is probably the correct thing to do, but boy is it going to be ugly...
So now you'll have to remember never to use 'execute()' but always 'execute_t()' or something.
You don't have to remember it, you can use deprecation warnings and lint tools to remind you. (Until eventually the safe API is the only API and then you really have nothing to remember.)
I’d assume their current safe function isn’t taking a string, and is taking some sort of prepared statement? So they could have it take either their prepared statement or a template, and deprecate their prepared statement.
If a library has functions taking a string and executing it as SQL they probably shouldn’t make that take a template instead, but I’d hope that’s a separate explicitly unsafe function already.
For sqlite3, it absolutely takes a regular string.
If you want to substitute parameters, you put a '?' in the string for each one, and provide an additional (optional) tuple parameter with the variables.
So no, there's no explicitly unsafe function. That's my point.
Gotcha. I’d guess they’d want to deprecate that function and create a new one that only accepts a template then, which is definitely annoying! I figured they’d already have more separation between prepared and raw strings which would make it easier.
Also I wonder how easy it will be to shoot oneself in the foot. It may be easy to accidentally make it to a string too soon and not get the proper escapeing.
That’s a library author problem, so it’s less likely since library authors tend to be fewer in number and, for popular libraries, get a reasonable number of eyes on this type of change.
Personally, this feels like a feature that is too focused on one problem to be a general feature. Python is getting huge. When people ask me if Python is easy and simple to learn I have to say "the basics, yes, but to to learn the whole language... not so much".
I feel like in this sense Go really is interesting by rejecting almost every single feature. Honestly not sure generics were worth it as they add a lot of complexity, and while they are nice, I don't need them very much. The general idea to keep the language at its original focus is the right idea IMO. C++ would be the most extreme case where the language itself barely resembles what it started out as.
Python has always been a batteries-included language, so having a go at templated string interpolation —a feature other languages have had for decades— seems like a strange gripe.
It's far more essential than little utilities like textwrap or goliath packages like Python's bundled tkinter implementation.
C# InterpolatedString is very close, with the twisty bit being that C# can rely on static typing for safety so the "f-string" and "t-string" variants use the same literal syntax and depend on what function they are passed to, whereas in both Python and Javascript they have different literal syntaxes. Python chose to use a different literal prefix to its literals ("f" versus "t") and Javascript chose to use a function-call syntax as prefix (`templateString` versus html`templateString` where html is a function in scope).
For the case like here it’s closer to FormattableString that’s what EF Core’s composable FromSql method works on top of. Both address custom interpolation but from different angles / for different scenarios.
Also, you can also find languages like Python that have a rich ecosystem of third party templating solutions. These are often intended for things like rendering entire web pages but many of them have relatively simple ways of using their templating functionality in a fairly reasonable amount of code, if you just want to have a template format that allows interpolating values based on a template string.
So, as you can see, many other languages have this feature, as you can tell from all the examples I have shown you here.
(To spell it out for those who may find this too subtle... somehow... I'm not a fan of this simply because Python has gone from "There should be one-- and preferably only one --obvious way to do it." to "there's half-a-dozen ways to do it and if they are all wrong Python 3.x+1 will introduce a seventh" and I'm just not seeing the benefits worth the tradeoffs here.)
I find this a confusing reply. First, you seem to be confusing eager string interpolation with this lazy/deferred templating feature. I'm specifically asking about languages that have the latter. More confusingly, you bring up the language under discussion as an example... if it already had this feature, they wouldn't be adding a new feature.
Then, you bring up 3rd party templating solutions, but this is off topic, as we are talking about language builtins. I am well aware there are many templating solutions to solve text templating in general.
In summary, I'm specifically asking for examples of languages that allow string interpolation with _deferred/lazy_ processing, which is how I understand this feature. It seems there are a few, but it doesn't seem to be terribly common after reading the other comments.
It's not confusing... it's sarcastic. I don't believe adding literally a third string interpolation method to a language is generally justified, and honestly you can consider it a fourth if you think, as I do, that even "my string " + val + " etc." is itself an interpolation language, especially when combined with things like __str__. I'm afraid you fed me a straight line I couldn't resist; it's not really an answer to you.
If I were in charge of Python, I'd probably feature-freeze it. It is already too large, but shrinking it is impractical.
Failing that, because it probably would fail, I would be raising the bar sky-high on new feature additions.
It seems like the project has actually lowered the bar in the last few versions. What was once one of my favorite languages, if not my favorite, is now a language I try to avoid, and approaching one I actively recommend against. When I'm writing it myself it's ok, but using other people's code in it is getting harder and harder, as they keep using all the features.
This is a pretty simple and useful feature. I wouldn’t say that it bloats the language too much. Descriptors and metaclasses are much more complicated and have a lot more implications and have been in the language for a veeeeery long time. Is it decades already?
This feature is not complicated, but one must keep every feature that can possibly be seen in code in their head. Even if it is familiar now, what happens when you use the feature in the one small section of code where it fits, nowhere else, and then read that code 2 years later? This is the problem with adding useful features that are only used in a few key places. I'm not saying Go is a perfect language, far from it, but limiting # of features as a general goal is something more languages should strive for IMO.
I am not arguing against that language ideally should have less features. I am arguing with “Python is getting huge”, because it’s huge and has been for many-many years :)
Yeah, Python hasn’t been a simple language for a long time, if ever. That’s probably the biggest misconception about the language - that its friendly syntax implies simple semantics. It’s not true at all.
I would say python in it's entirety is one of, if not the deepest and potentially most complex language I know. C++ is the other contender. The things you could do with metaclasses, multiple inheritance and operator overloading are quite staggering.
I'm just glad you don't have to think or even use this as a normal user of the language, most of the time or at all.
A lot of Python's complexity like descriptors and metaclasses and the so-called cooperative multiple inheritance come from an era where object orientation is gospel and yet people keep finding limitations to what can be accomplished in an OO paradigm and therefore they patch up the language to support ever more complicated use cases.
I have a feeling that if metaclasses and descriptors were to be proposed today it would be laughed out of the PEP process completely.
> I have a feeling that if metaclasses and descriptors were to be proposed today it would be laughed out of the PEP process completely.
I think you need at least one of these. Modern Python eschews metaclasses but descriptors are everywhere. Without them you couldn’t have @classmethod, @staticmethod nor @property - or at least, you couldn’t generalise custom method lookup and all three of those would need to be special cases.
The alternative approach is to drop descriptors and make method lookup much simpler, but then you need metaclasses (see: static methods in Ruby).
If descriptors didn't exist in the language, classmethod, staticmethod, and property would become keywords. Languages like Java and C++ already have static as a keyword; C++ also made its type system have a special pointer to member type; such behavior can be hardcoded without having a flexible descriptor mechanism.
Custom method lookup would just be the same as custom attributes, that is, by the user overriding __getattr__.
I'm truly glad that Go exists for people who like languages that are simple even to the point of frustration and I hope it never changes. But I'm glad that other languages exist for those of us for whom learning some syntax is not a barrier and having convenient ways to do common things is highly valued.
There’s an interesting trade off around maintenance. Python’s stdlib means there’s more consistency across projects and you can assume basic things like error handling which you had to check individually on each Go program, which is the kind of stuff which adds up when you have a lot of small programs.
This is especially noticeable with AWS Lambda where you can have a lot of useful stuff running for years without doing more than bumping the runtime version every year or two, but also that is one highly opinionated architecture so it’s not globally optimal for everyone.
This is exactly how Nim is. The f-string like equivalent uses a macro called "fmt" which has a short alias "&". So you can say:
import std/strformat
let world = "planet"
echo &"hello {world}"
The regular expression module does a similar thing with a `re"regular expression"` syntax or std/pegs with peg"parsing expression grammar" and so on. There are probably numerous other examples.
In general, with user-defined operators and templates and macros, Nim has all kinds of Lisp-like facilities to extend the language, but with a more static focus which helps for both correctness and performance.
This was the original proposed idea in the PEP (750), but it changed overtime. There is a section in the PEP to explain why it changed to t-strings if you are interested.
PEP 638 has always seemed to me like something of a generalization of the idea. But that really feels to me like a 4.0 feature, or rather something that needs to be designed for from the beginning. (Which is why I've done a bit of that in my own mind...)
No, you misunderstood that completely. They would still be like t-strings, but sql-strings are now a different type from html-strings. The escaping would be done by the library that offers the sql"..." functionality.
FWIW the JS equivalent is a template but tagged with a language. It has all the benefits of this template, but IDEs can easily syntax highlight the string. That seems like it would be a bit trickier to do with the Python one which is a shame.
In my own language design (nothing public yet - need to prioritize more practical things at the moment) this is definitely on the menu. Aside from a minimal set, keywords are implemented as compile-time macros; and it's intended that they can be extended, both "natively" and in the implementation language, by writing AST-manipulation code. But the handling of arithmetic expressions, as well as the broader line/block structure, is hard-coded. (I draw inspiration from both Python and the Lisp family.)
What I really don't get is how it's any different than applying whatever function you would apply to the template, on the f-string variables. So instead of:
> Is it just about not forgetting the sanitization/string manipulation part and forcing you to go through that?
This is a very big deal! It's also about centralizing that work. Now that sanitization can occur in the consumer of the t-string (for example, the API to your HTML renderer), rather than in every f-string.
Pretty much, yeah. The article highlights that people were using f-strings directly, and they wanted to provide an alternative for lightweight template/interpolation.
I feel like I'm still missing something when they're saying this about the example(s):
"Neither of these examples is possible with f-strings. By providing a mechanism to intercept and transform interpolated values, template strings enable a wide range of string processing use cases."
As far as I can see, anything you do with the template, you could do before building the f-string or inline as in my intial example.
With f-strings, you cannot write code to determine which portions of the resulting `str` were static and which were dynamic; with t-strings, you can. *
(As to your initial example: it's worth considering what will happen as you compose multiple bits of HTML via nesting to generate a large final page. The developer experience may become... unideal.)
You wouldn’t really do your example, though. If you’re using an f-string, you’d just directly interpolate, because it’s convenient. You wouldn’t use an extra library to properly make it safe, otherwise you’d just use a proper template library and language.
This gives you a convenient middle ground where you don’t need to learn a template library but still get safety. I can’t think of the code right now but I could see this being useful to pass in some dynamic HTML to, say, Django without having to remember to turn off escaping for that section. It can also be convenient for writing raw SQL without having to use prepared strings.
I see each of these as distinct but overlapping; I'm (slowly) writing a guide to string formatting with all of these in mind, trying to emphasize when I'd choose one over the other. (fwiw I personally avoid % and + these days; $ is pretty uncommon in practice; f-, t-, and .format() all seem to have good unique uses.)
The last three generally shouldn't be used (`+` is sometimes fine, but not really for formatting), but I doubt we would ever get a py4 that removes them, given the stomachaches that py3 caused. It does feel odd that a t-string is just an f-string without args applied.
It's my understanding that they're still recommended for logging since with f-strings you always pay the formatting cost (but with str.format it's deferred).
I could scarcely believe this new t-string wasn't a joke. As an occasional, grudging Python programmer, I rue the accretion of string formatters over the years. My code bears the history of successive, imperfectly understood (by me)formatters.
Why does this need to be a language feature. This could just be a separate library, we could use brackets instead of a letter before a string. I fear, Python is going down the path of C++
It being a language feature gives you controlled access to the lexical scope, such that the template string can refer to variables by name rather than having to pass each value as a parameter. Doing it via parameters is repetitive and/or error-prone.
This feature actually can't be a library. It needs to be able to preserve the string before the variables are passed in, which a library would be unable to do because f-strings immediately replace all the values.
The whole premise of t-strings is to know which values where "hard coded" and which ones are variables. A library can't possibly know that. And a function call wouldn't have access to the local variables.
The only way to do this without language support is to pass `locals()` into every function call.
Now you have me wondering how difficult it would be to have a class that takes a string and parses through globals to find the context it was called from. maybe causing an exception and abusing a traceback? or maybe we just find our own objectid.... gahh I have to try it now, but I'm setting a timer.
It was easier than I thought, but also took longer than I wanted to. turns out the inspect module provides what you need to pull it off.
This dummy example splits a string that it was given, then if one of those values is in the callers context it saves those in self.context, and has an output function to assemble it all together. Obviously this example is not very useful, but it shows how a library could do this as a class or function without the user having to pass in locals().
import inspect
class MyXString:
""" will split on whitespace and replace any string that is a variable name
with the result of str(variable)"""
def __init__(self, string):
self.string = string
caller_locals = inspect.currentframe().f_back.f_locals
self.context = {}
for key in set(string.split()):
if key in caller_locals:
self.context[key] = caller_locals[key]
def output(self):
output = self.string
for k,v in self.context.items():
output = output.replace(k,str(v))
return output
This looks like a performance nightmare, and would likely never have IDE integration. So I guess I was wrong. It could be a library. But will be much better supported officially.
You don't have to go through all the globals, you just have to get the caller's namespace, which is fairly simple. See e.g. https://stackoverflow.com/a/6618825/27426
For this reason, I think it's not true that this absolutely had to be a language feature rather than a library. A template class written in pure Python could have done the same lookup in its __init__.
It's not as simple as "more features" == "closer to C++". Features are not equal to each other in terms of impact on language complexity.
t-strings don't interact with anything else in the language; they, as you yourself pointed out, could almost be an isolated library. That makes them low impact.
This is also true syntactically; they're just another type of string, denoted by "t" instead of "f". That's easy to fit into one's existing model of the language.
Moreover, even semantically, from the point of view of most language users, they are equivalent to f-strings in every way, so there's nothing to learn, really. It's only the library writers who need to learn about them.
Then we have to consider the upsides - the potential to eliminate SQL and HTML injection attacks. The value/cost is so high the feature a no-brainer.
If it's not a language feature, there's always a risk of fragmentation. Some people won't use it because it adds another dependency, that means fewer programmers will be familiar with it. Others will come up with their own, slightly incompatible implementation. See for example Perl and its various Object Orientation frameworks (Moose, Mouse, Moo, Mo, M, Mojolicious comes with its own...)
Other languages have a policy of prototyping such things out of core, and only adding it to the core language if it gains traction. Of course that works better if the language has a mechanism for extending the syntax out of core.
I'm a little late to the conversation (and a bit surprised to see this trending on HN) but am happy to answer any questions; I'll try to pop in throughout the day.
The PEP originally started with a similar-to-javascript syntax but over time we decided it wasn't the right way to expose these ideas in Python. There's more detail about why this approach was rejected in the PEP: https://peps.python.org/pep-0750/#arbitrary-string-literal-p...
PEP 292's `string.Template` will remain; there are no plans to deprecate it.
PEP 750's `string.templatelib.Template` is a separate and unrelated type. Amongst many differences, unlike PEP 292, `Template` has a literal form too.
I'm hopeful that the confusion will be minimal; in practice, PEP 292 (aka $-strings) is used only in specialized cases, like flufl.i18n, a really deep I18N framework.
Landing in 3.14? Nice, but also oof, that's probably not getting to my employer's codebase for a year or two. And it sounds like it could really solve some problems for us, too.
Paging asottile - any plans to make a `future-tstrings`? :)
I asked asottile for his opinion on PEP 750 (or rather, a previous version of it), and he wasn’t really in favor. So I think it’s unlikely that we’ll see a future-tstrings from him. :)
How do these interact with i18n? Can I load a translated t-string with `_(t"")` from a .po file? Can it include variable names and arbitrary code inside lambdas?
> t-strings evaluate to a new type, `string.templatelib.Template`
> To support processing, `Template`s give developers access to the string and its interpolated values before* they are combined into a final string.*
Are there any use-cases where processing a Template involves something other than (i) process each value, then (ii) recombine the results and the string parts, in their original order, to produce a new string? In other words, is the `process_template` function ever going to be substantially different from this (based on `pig_latin` from the article)?
def process_template(template: Template) -> str:
result = []
for item in template:
if isinstance(item, str):
result.append(item)
else:
result.append(process_value(item.value))
return "".join(result)
I haven't seen any examples where the function would be different. But if there aren't any, it's strange that the design requires every Template processing function to include this boilerplate, instead of making, say, a `Template.process` method that accepts a `process_value` function.
Templates don't even have to be processed into a string. The article shows an example where the Template is processed into an HTML mini-DOM. It's maybe not obvious because the DOM object is immediately stringified to show sample output, but you could imagine manipulating the DOM object in a few more steps before stringifying it, or maybe you are running in WASM in a browser and using that mini-DOM directly as a Virtual DOM passed to JS to work with.
Also, in addition to the other SQL example using "?" to fill in the "holes" for parameters in an SQL friendly way, some DBs also support named parameters, so the "hole" in the string form might be naively replaced with something like `f"@{item.expression}"` and that also forms the key in a dict to pass as parameters. (You'd want to make sure that the expression inside the template is useful as a parameter name, and not something more exotic like {1 + 3} or {thing for thing in some_list}, in which cases you are probably auto-assigning some other parameter name.)
Also, my comment was about the amount of boilerplate required, but that can be vastly reduced by writing `process_template` in a more functional style instead of the highly-imperative (Golang-like?) style used in the article. The first `process_template` example is just:
>But if there aren't any, it's strange that the design requires every Template processing function to include this boilerplate
Other replies gave examples of other use cases. But the neat thing about Python is that you don't need to "include this boilerplate" for the common cases. It can be wrapped up in a decorator (which could be included in `templatelib`). Or, as you say, in a method on the Template class.
I think I'd implement it as a generator, calling `process_value` (defaulting to `str`) on the Interpolations, so that the caller can still do more with the results (or just `''.join` them).
But these are separate considerations; nothing prevents implementing them later, or indeed adding them to the implementation before the 3.14 release.
I wish they added the same thing JS has, where this "string literal prefix thingy" can be user-defined.
html`<p>${value}</p>` will actually run the function html(template). This means you can use this to "mark" a function in a way that can be detected by static analysis. Many editors will, for example, syntax highlight and lint any HTML marked this way, same with SQL, GraphQL and probably some others too.
The PEP was initially proposed this way. But due to various reasons, making it an open namespace was considered to overcomplicate the language (read: understanding coffee when reading it). Alternatively, there doesn't really seem to be much loss of ability with t-strings. Libraries can require a template as it's accepted type, instead of having to invent their own custom type and named template.
For the record the JS thing desugars to the exact same as the Python thing, so it is no more or less safe to do the syntax highlighting in Python as it is in JS.
It desugars similarly, but the Python version doesn't have a name. Any t-string is a t-string, there's no HTML t-string or SQL t-string or anything like that. It's just a t-string you can pass to a function:
html_string = t"<something />"
sql_string = t"SELECT * FROM something"
In JS, the string has a prefix that can differ between languages, e.g.:
Except your labels are incorrect because neither `html_string` nor `sql_string` are strings, they're both Template objects, and the sink function is the one which processes it. No processing has happened to them by the end of the snippet, beyond creating the template object itself.
Sure, choose different variable names, who cares. The essential difference is that the language is referenced at the declaration site, not the usage site, which makes the syntax highlighting far easier.
Please engage with my point instead of criticizing trivialities.
> Please engage with my point instead of criticizing trivialities.
Your complete misunderstanding of what's happening is not a triviality.
> The essential difference is that the language is referenced at the declaration site, not the usage site, which makes the syntax highlighting far easier.
Javascript has no built-in template tags beyond `String.raw`. If tooling has the capabilities to infer embedded language from arbitrary third party libraries, I would hope they have the ability to do utterly trivial flow analysis and realise that
html(t"<something />")
means the template string is pretty likely to be HTML content.
Yes. It's a weak heuristic, but it's EXACTLY the same weak heuristic that JS is applying!
IN other words, since custom template tags in JS *are literally just function calls* when a JS environment syntax highlights the code as HTML it's doing so based on an extremely weak heuristic (the identifier for the interpolation function is named "html"). Both Python and JS have the same problem.
I think his point would be clearer if we focused on typing the usages statically. Consider `html(this_is_a_sql_template)` vs. `html"SELECT * FROM ..."` or `thing_that_consumes_html_template_type(oops_a_sql_template_type)`.
Come on, you're just being rude for no good reason. A badly chosen variable name doesn't show a "complete misunderstanding". Yes, the variable should have been named `html_template` instead of `html_string` - how often do I have to acknowledge this before you accept it?
And it's obviously more complex to do syntax highlighting when the declaration site and usage site are possibly split apart by variable assignments etc. Yes, in the case you showed syntax highlighting is easy, but what if the `html` function takes more parameters, doesn't take the template as the first parameter, etc? There's a lot of possible complexity that tagged template literals don't have. Thus they are easier to do highlighting for. This is objectively true.
Tagged template literals in JS have all that complexity. All of it. That tools you trust lie and pretend that's not the case doesn't make the language spec say anything different
No, they literally don't, because they don't support these features!
You can't split apart the declaration of the template literal and the "tagging". The tag is always part of the declaration, which it doesn't have to be in Python, as I've showed.
You can't pass additional parameters to the tag function, it's always just the template & values. In Python, you can pass as many parameters as you want to the usage site, e.g.
It just sounds like you don't know JS very well, because in JS you can definitely split apart the declaration and the tag with ease. The thing implementing the tag is just a function after all: https://jsfiddle.net/rd2f1kot/
you already have a string in the `template` variable. There's no access to the individual values anymore. It's already done. It's a string. No template. You can't pull apart the literal declaration and the tagging.
Are we now done with this ridiculous game of deliberate misunderstandings?
Sorry for some reason I can't reply to the deeper posts. Depth limit I guess. You asked which editors could support syntax highlighting once the tag and the content that needs to be syntax highlighted are split apart. Currently there are none, though I am writing one that will be able to.
Python should be able to detect the magic syntactic pattern just the same way JS does and use that for syntax higlighting. In JS the magic syntactic pattern for triggering HTML syntax highlighting is:
html`<doc/>`
In Python the magic syntactic pattern would be:
html(t"<doc />")
My point in showing that JS counterexample was to demonstrate that the real reason people don't do that kind of thing isn't that they can't, it's that they like having syntax highlighting. That means the approach should work just as well in Python. This is the case even though the heuristic involved is very weak. Changing the name of the identifier or breaking into multiple expressions would be enough to break the heuristic in either language, which is why I think it's a really weak heuristic and dangerous pitfall for developers who might mistake the coloring for information about how the runtime sees that data (which is normally what syntax highlighting is for)
If I may give you some advice: you could have made this point without telling me I don't know JS well, without misrepresenting me and without wasting both of our time by just making this point from the start. Because you are technically correct that, in the limited case of a template immediately being passed to a function, syntax highlighting can use the same heuristics that are currently used in JS. But, as I assumed previously, in doing so you're simply ignoring my point: syntax highlighting for tagged template literals isn't applicable to only a subset of uses because they are by design less complex. Your counter example was using a different template tag and not using the actual tag as a template tag, which obviously doesn't work in the context of syntax highlighting for that template tag.
Had you not played games with this technicality, we could both have saved a lot of time. Hope you had fun I guess.
So now we have moved from "tagged template literals have all the complexity of Python templates" to "if you define a specific template tag and use that, you can replicate one feature of Python templates". No mention of the other example I mentioned (which, by the way, wasn't an exhaustive list).
Now, I'll just ignore that you're still deliberately misrepresenting me in the hopes of gaining some actual knowledge from this discussion - please show me: which editor supports syntax highlighting for the specific example you just mentioned? After all, that was the topic of discussion - not whether it's possible to delay "tagging", but whether it's easier to do syntax highlighting for JS template tags. Please show me an existing editor, IDE or git repo for a tool that would highlight your specific example based on the `html(...template)` call in line 4 (so no heuristics based on line 3). Surely you're not just throwing random examples at the wall to see what sticks, right? You've actually been following the discussion and are arguing in the context of it?
No, it doesn't, it references the language at the declaration site, because the declaration site always is the usage site. You can't split them apart. You can split them apart in Python - see the example in my first comment.
The JS version actually desugars to something much more primitive, and less convenient: in JS a template tag receives one parameter which is an array of n+1 strings, and then n parameters for the interpolated values, and has to iterate alternatively on both sequences.
You can do that in python by accessing the `strings` and `values`, but I expect most cases will simply iterate the template, yielding a unified typed view of literal strings and interpolated values.
> If you’ve worked with JavaScript, t-strings may feel familiar. They are the pythonic parallel to JavaScript’s tagged templates.
The syntax is template literals, not just "tagged templates". Which is a huge difference: template literals still act as real strings. They don't need a tag prefix to work, you have the option to tag them if and when needed.
As far as I understand it, t-strings can't do that. They're not strings, and you can't even coerce them into strings, you have to run them through a processor before they become a string. So they're nothing like JS's template literals, they're syntactic sugar for forming "an instance of an object that needs to be passed into a function that returns a string".
So I don't look forward to folks preferring f-strings over t-strings even when they really shouldn't, simply because "having to constantly convert them from not-a-string to a string is a hassle". If only they'd worked like JS template literals.. that would have been fantastic.
Debate around the usefulness aside, are there any linter rules for warning about f-strings in light of this? I can easily see where mistaking one for the other would cause problems. For context, I'm thinking specifically about tools like Black and MyPy.
The runtime and static type of the template string is different to fstring (Template vs str). Unless you’re passing them somewhere that accepts strings, you can’t go wrong.
I actually quite like the simplicity of this design over tagged literals in JS.
No, I mean that if Django's template engine internally switched over its implementation to enable templating (as a functionality) to use these new t-strings. If t-strings are more highly optimized within Python, it could mean a performance boost.
TL;DR: like f-strings, all {foo} expressions in the t-string are evaluated immediately, but instead of immediately concatenating everything into a single result string, the t-string evaluation returns a Template object that keeps the interpolation results and the surrounding strings separate. This lets subsequent logic decide whether the interpolation results need any special escaping before concatenating them with the strings around them.
In other words, t-strings are basically f-strings where the final concatenation is delayed. And indeed, you can trivially implement f-strings using t-strings by performing a simple, non-escaped concatenation step: https://peps.python.org/pep-0750/#example-implementing-f-str...
> This lets subsequent logic decide whether the interpolation results need any special escaping before concatenating them with the strings around them
This sounds like unnecessary fluff in what was supposed to be a simple language. I'm worried Python is turning into C++42 with 65535 ways to do one simple thing.
Why not just:
f'SELECT * FROM `{esc(table)}` WHERE name = "{esc(name)}"'
It's easy to forget the `esc` function. How does the recipient check (or type check) that it was called in all the right places?
Most DBs support parameterized queries which can be cached for performance. How do you pick out the parameters from that and replace those parts of the strings with the DB's parameter placeholders?
t'Select * from {table} where name = {name}'
Looks very similar, but execution engine has access to all the individual parts, making it very easy to add placeholders such as:
('Select * from ? where name = ?`, table, name)
Or even (if the DB supports it), has access to the expressions inside the string and can use named parameters:
('Select * from @table where name = @name', { "table": table, "name": name })
That's really nice for debugging, depending on your DB engine.
In every DB engine that supports it, parameterized SQL is even safer than escape syntaxes because parameters are passed in entirely different parts of the binary protocols and don't need to rely on just string manipulation to add escape sequences.
While your code is a valid alternative way to implement @haberman's description, the feature is actually much more flexible.
The "subsequent logic" has full access to the interpolation results and strings. Not only can it escape the results, it can do whatever it wants to them. It can also do whatever it wants to the strings, and then combine everything in any way it likes - it's not even necessary that the final result is a string.
The other PEP example shows generating HTML attributes from a passed-in dictionary. HTML has a number of places where this is helpful, if you have original data.
by making it a generic `t` you lose explicit syntax highlighting. Where something like JS template`string` could determine which syntax to use based on the template value.
I supposed when assigning it to a, variable: SyntaxRecognizableTemplate, you could give it the hint necessary.
was this discussed in the PEP?
*edit: reading the PEP-750[1] it doesn't seem like it..
But yes, the PEP leaves open an important question: how will tools decide to work with common types of content in t-strings, like HTML or SQL?
There are simple approaches that can be taken in the short term (content sniffing) and more robust approaches (type annotations, perhaps) that will take time and the broader tooling community to develop.
It was discussed in the first revision and discussion of the PEP. The decision was made to move that to follow-on work, as we discovered more about what tooling needs.
As an example, I was excited about using `Annotated` on the function to indicate the language it expected. Turns out, a lot of linters know nothing about the type system.
Not sure about introducing yet another string prefix. Between f-strings, raw strings, and i18n stuff, it’s already getting crowded. Curious how readable this will be in large codebases.
How would this be different from a function sql() that operates on one of these new t-strings?
The syntactic sugar of changing it from sql(t"...") doesn't seem particularly valuable. The novel thing about t-strings is that they change the parsing at compile-time.
It’s different from a function the same way f”” is different from f(“”) and t”” is different from t(“”)
There’s nothing stopping you from building a Python function that parses a string looking for {} and then searching globals for those variables. And you can extend that to also do some code execution and formatting.
To me the real sugar of f-strings is that the editor knows that it’s a template and not just a string. Expanding this to having SQL and regex syntax highlighting, linting and code formatting inside my Python code is a pretty cool prospect.
Additionally, it will probably be confusing that it is called a t-string but it is actually a constructor for a Template object and not string at all. I would rather see a new special term `template` than this.
No. F-strings should still be the default. T-strings aren't usable directly, and don't have a `str` like API, so they aren't even compatible, intentionally.
Honestly think this is a more useful feature and elegant solution than the walrus operator that was added. Formatting query strings has always felt messy especially with different DBs having their own non-standard ways of doing it.
I'm intrigued but confused by your comment. I think it could be done like this:
cur.executemany(t"INSERT INTO movie VALUES({data})")
Then the Template object would have a parameter that is a list, and it could turn that into the right number of "?"s to pass into the database driver along with the data.
Seems pretty neat so far, but I don't understand the motivation behind not allowing you to call str(template) and get the template as a normal string. I could imagine it being very useful to be able to gather up the template itself in a string to do stringy things with.
The only reason I could imagine, is if you are trying to protect developers from themselves, which kinda goes against the "we're all adults here" mentality that makes Python so great. I suppose it's easy enough to add that functionality, but come on.
I don't think things would be very different if Guido were still BDFL. He's second-author of the t-strings PEP, and has been an author of most other major PEPs in recent releases (including the walrus operator, PEG parser, and pattern matching).
It's so fascinating to see so many people, when faced with a simple and useful programming language feature, get all up in arms about how it's the end of the world for that language.
I honestly feel like a lot of people just seem bored and looking for stuff to be mad about.
It’s injection safe and compostable, and the resulting object retains the original values you interpolate in. This makes it useful for building SQL queries with prepared arguments for example.
It’s on a meta level: instead of formatting the string, it returns an object that contains both the format string and its argument. Library author can then implement whatever format function they want, for example one that escapes the interpolated strings.
f"foo is {foo} and {bar=}"
"foo is {} and bar={}".format(foo, bar)
are equivalent.
t-strings are actually not strings, but Template objects, giving access to both the templating string and the parameters for processing. Sibling comments describe it as a custom .format implementation in that sense - it's f-string-like sugar where you're also allowed to take control of the .format function that it's sugar for.
f-strings are actually translated into formatting and concatenation of the pieces at a bytecode level; they don't call `.format` under the hood and thus aren't what I'd call "syntactic sugar" in the traditional sense. But yes, the equivalence holds.
t-strings, of course, (will) translate at the bytecode level into instantiation of the Template object (and whatever code is needed to compute the Interpolation values in-place).
If this is just for sql queries ... it'd be overkill especially where you need to compare the usual PREPARE statements with the hassle of keeping everyone on 3.14 and above.
Sure, this avoids issues with SQL injections. However, I have a hard time imagining any developer who would both make such fundamental errors with f-strings currently and also switching to this option when it ships.
Seems like a self selection which renders this meaningless, to some extent :/
> However, I have a hard time imagining any developer who would both make such fundamental errors with f-strings currently and also switching to this option when it ships.
t-strings are a different type (both static and dynamic), f-strings are not. So t-strings can be mandated at the API level, forcing the developer into "proper" usage.
That is, you need third-party tools to differentiate between
some_call("safe literal string")
and
some_call(f"unsafe dynamically created string")
That is not the case when it comes to t-strings, `some_call` can typecheck internally that it got a t-string, and reject regular strings entirely.
Although some space probably needs to be made for composing t-strings together in case of e.g. runtime variation in items or their count. Facetting for instance. I don't know if that's a native affordance of t-strings.
sqlalchemy doesn’t really accepts strings - if you do, you need to pass them into a “conn.execute(text(…))”, so end users should not face a breaking change.
I feel like this can be solved another way. S=f”my good code #### {potentially_evil_user_input} #### my good code again” then work around the ####. Of course, even better, S=evil_user_input and do a scrub on S first.
I really was on the side of being generally willing to accept new python features, but this is getting ridiculous. What an utterly pointless thing to bloat the language with. At this point my moving to clojure as my first line language of choice is only accelerating.
This is of the category "things I wouldn't want to use even for the specific hyper niche things they're intended for". What even does a "t-string" represent? Because it's clearly not a string of any kind, it's a weird kind of function call notation. The programmer sees something that looks like string formatting, but the program executes some arbitrary procedure that might not return a string whatsoever.
For me, this is the best feature to land in python for 6 years. JS has had this and it allows my code to be completely safe from SQL injection, which is an absolutely incredible feature, given SQL injection has been the #1 vulnerability for a long time.
Thanks Nick for this response and all the time you've spent explaining. It's funny, I looked back at the comments on f-strings before they landed. They also got similar complaints about bloat. And yet, my uneducated guess: very popular.
But this doesn't prevent SQL injection, does it? It adds a grammar feature that you can then use to build SQL injection prevention following a novel idiom. Someone still needs to write the actual SQL building logic somewhere!
I don't think this is the right idiom for doing this. Frankly I don't think SQL query generation should look like string templating at all!
The sell seems to be "now you can write code that looks like an SQL injection vulnerability, without it actually being vulnerable!". I'd rather write code that isn't a vulnerability, and doesn't look like one, and doesn't have to bend the language grammar either.
I'd rephrase the sell as "now people who are used to doing the wrong thing and risking vulnerabilities can do the right thing without any extra effort," with a footnote about the difference in types allowing libraries to force the change.
What? Maybe in 2005. This is just strictly worse than parameterized because now you are also wasting time "escaping" strings which perpetuates the whole mixing data & query thing nobody even wants anymore.
It's like even the one case identified nobody has even thought all the way through. Now your SQL library only accepts t-strings, I get an obscure error passing in a simple static query. Ah yes, put the useless t on it. That sorted, now the SQL library escapes all the parameters it wasn't previously doing, to then hand the final unique query to the actual underlying SQL library which would much rather have the parameterized one so it can cache parsing. Jesus.
I'm not the OP but I'm guessing with OP moving to Clojure that the bloat is basically any special syntax or special language feature that basically boils down to a function call or a let statement. A lot of functional programming languages have minimalistic expression syntax and they are just as expressive as OP needs them to be.
It's a pattern moreso than this specific feature. There's an important qualitative distinction for me between something that's on the level of a library (even if it's the stdlib) on the one hand and first class features (ie things that require parser level support) on the other.
Python has historically been very conservative about this but in recent years has had one controversial language extension after another, while parts of the language that actually need love are left to languish IMO.
I wanna be very clear that this is me changing my mind -- I was (still am) very on board with the highly controversial assignment expressions ("walrus operator") for instance.
I don't have much faith about what the Python language will look like if you project the current rate of changes forward 10, 15, 20 years. It really doesn't help that I consider this new thing an active antifeature.
I feel like this is begging the question... This t-string feature is only controversial inasmuch as comments like yours here are criticizing it as being controversial...
I enjoy f-strings, I guess some people need these.
And I love Python but, having been through 2->3 ( occasionally still going through it! ) whenever I see a new language feature my first thought is "Thank goodness it doesn't break everything that went before it".
Yeah but it's been 17 years, maybe it's time to put the PTSD behind us. We're almost at a point where the current generation of programmers wasn't even programming when that happened.
> We're almost at a point where the current generation of programmers wasn't even programming when that happened
I've been programming with Python since 2006, I think most of the systems were based on 2.4 at the time. I've been one of those who switched to Python 3 somewhat late, waiting for some major libraries to ship python 3 packages - celery and Twisted were one of the biggest holdouts - so I remember that the first project where all my dependencies were ready for python 3 was around 2015.
This is to say: even seasoned developers who were conservative around the migration have spent more time working with Python 3 than Python 2. There simply is no reason anymore to be talking about python 2.
The last time I touched a large Py2 project was in 2018 when I ported it to Py3. So, I have 18 years of Py2, probably 6 years of overlap, and 7 years of pure Py3. That means I still have a lot more Py2 than Py3 time.
Buuuttt, I'm so over the transition. It’s ancient now and I agree that we can stop fretting about it.
Python2 code didn't disappear when Python3 came out. At my work we're _still_ occasionally having to help people migrate code that was written for python2
And Ubuntu ESM got used as an excuse/"life support" for python 2 via 16.04 until horrifyingly recently. (With a layer of "you can still get ESM for 14.04, we're not that far behind" :-)
All things considered, this is pretty cool. Basically, this replaces
with Does the benefit from this syntactic sugar outweigh the added complexity of a new language feature? I think it does in this case for two reasons:1. Allowing library developers to do whatever they want with {} expansions is a good thing, and will probably spawn some good uses.
2. Generalizing template syntax across a language, so that all libraries solve this problem in the same way, is probably a good thing.
I did a safe OCaml implementation of this about 20 years ago, the latest version being here:
https://github.com/darioteixeira/pgocaml
Note that the variables are safely and correctly interpolated at compile time. And it's type checked across the boundary too, by checking (at compile time) the column types with the live database.
Yes, what you did is strictly more powerful than what the Python people did. And you did it 20 years ago. Well done, have an upvote. And yet, here we are in 2025 with Python popularity growing unstoppably and (approximately) no one caring about OCaml (and all the other languages better than Python). It makes me sad.
Network effects are a beast!
But my two cents is that we're pretty lucky it's python that has taken off like a rocket. It's not my favorite language, but there are far worse that it could have been.
You mean like Cobol? Oh wait!
I'm switching between C, OCaml, Python, bash & Rust roughly equally every day (to a lesser extent, Perl as well). Not everything is what gets on the front page of HN.
It's interesting how the majority has explicitly chosen NOT to use the "better" languages. Is the majority really that bad in their judgment? Or is it that "better" is actually defined by adoption over time?
It's clearly better in their opinion, they just aren't optimizing for the same metrics that you are. Python is better because it's easy for people to learn, imo.
its not easy to learn. its a challenge even getting it installed and running. what even is a venv? how do you explain that to a beginner?
python is popular because its what teachers teach.
If someone is challenged figuring out a venv and they're not an absolute beginner, perhaps they aren't cut out to work in technology. There are countless subjects in the field more challenging and complicated to wrap one's brain around.
Also, in 2025, just use uv.
> they're not an absolute beginner
gp's claim is not "its easy to learn". It's not just the concept -- it's the ergonomics, absolutely terrible footguns (especially when dealing with global wheels that can screw up your running system), and the hidden state.
When would you want to interface between a specific project and the global Python environment running your system? If there's ever a time when "lock the project into a venv and don't cross-contaminate its dependencies with the global Python environment" isn't the answer, that sounds like a corner case. Let each project be its thing by itself.
On modern Linux you can type `python` at the command prompt and get a REPL. On Windows you download an installer from the official website (just like one usually does to install anything on Windows), then use `py` at the command prompt.
You don't need to `import` anything to start teaching Python. Even then you can do quite a lot with the standard library. Even then, unless you're using 3.11 or later on Linux you can let Pip install with `--user` until you actually need to isolate things between projects. (And even with new Python on Linux, the instructor can typically avert this by just installing a separate Python in `/usr/local/bin` for example. Yes, that's "cheating", depending on the classroom environment. But that's part of the point: installation hurdles are hurdles for self-learners, not for students.)
You only need to learn about virtual environments once you have projects with mutually conflicting dependencies, and/or once you're at a point where you're ready to publish your own software and should be learning proper testing and development practices. (Which will be largely orthogonal to programming, and not trivial, in any language.)
And when your students do get to that point, you can give them a link such as https://chriswarrick.com/blog/2018/09/04/python-virtual-envi... .
Teachers teach Python because it's easy to teach while still being relevant to the real world, in particular because boilerplate is minimized. You don't have to explain jargon-y keywords like "public" or "static" up front. You don't have to use classes for quite some time (if ever, really). You can express iteration naturally. Types are naturally thought of in terms of capabilities.
In my mind, Python has all the pedagogical advantages of Lisp, plus enough syntactic cues to prevent getting "lost in a sea of parentheses". (Of course, it lacks plenty of other nice Lisp-family features.)
> In my mind, Python has all the pedagogical advantages of Lisp, plus enough syntactic cues to prevent getting "lost in a sea of parentheses". (Of course, it lacks plenty of other nice Lisp-family features.)
What you say here reminds me of something Peter Norvig said 15 years ago on this site: https://news.ycombinator.com/item?id=1803815
> Peter Norvig here. I came to Python not because I thought it was a better/acceptable/pragmatic Lisp, but because it was better pseudocode. Several students claimed that they had a hard time mapping from the pseudocode in my AI textbook to the Lisp code that Russell and I had online. So I looked for the language that was most like our pseudocode, and found that Python was the best match. Then I had to teach myself enough Python to implement the examples from the textbook. I found that Python was very nice for certain types of small problems, and had the libraries I needed to integrate with lots of other stuff, at Google and elsewhere on the net.
Basically, that it's better pedagogically because it looks like pseudo-code and it's easy to get up and running quickly.
Which is valid, but frustrating to see it lead to actual adoption outside of pedagogy. That property is entirely orthogonal to, almost at odds with, what makes a good programming language for medium to large production quality applications.
If we used that logic elsewhere in life we’d all be playing the flute and cycling around on tricycles and balance bikes. But for some reason in tech it’s all about Hello World.
The story of the winner being scrapy market entrants that are lower-cost (...of learning, in the case of python) and good-enough-quality (...than OCaml, Lisps, Haskel, definitely not JS or Java) is not a new one. I don't subscribe to your analogies.
> You don't have to explain jargon-y keywords like "public" or "static" up front.
patently not true. you dont get too far into python -- especially if you are reading (or copypastaing) other People's code -- before you see if __name__ == "__main__" and any potential future programmer will rightfully ask "what the absolute fuck is this"
even "def" is kind of a weird fucking keyword.
Don't get me started about teaching beginners which datatypes are pass by reference and which are pass by value.
try explaining to an elementary school student why
doesn't change the caller's variable but does.> Don't get me started about teaching beginners which datatypes are pass by reference and which are pass by value.
If they are beginners to programming, you wouldn't teach them those terms in the context of Python, because neither of those terms map to Python argument passing; Python has one form of argument passing, and it doesn't map closely to the intuition that experienced programmers in languages that have pass by reference and pass by value have about those things. Likewise, the only thing you'd teach someone new to Python that is experienced in languages where those terms are useful is that that distinction is irrelevant in Python, which is pass by assignment (sometimes also called pass by object reference, but pass by assignment is a much more useful description IMO, because argument passing works exactly like assignment to a new variable name.)
> try explaining to an elementary school student why
> doesn't change the caller's variable but > does.But, that's easy, if you've first taught them how variables, assignment, and mutation work in Python without getting function calls in the way, because it is exactly the same as this
vs. Argument passing is just assignment to a new variable that exists in the scope of the function. Methods which mutate an object affect the object no matter what variable you access it from, assignment operations affect only the variable they assign to. That's exactly the same behavior in one scope as it is between the function and caller scopes.And this distinction has nothing to do with data types, but only with the operations performed (the only connection to data types is that immutable types have no mutation operations in the first place.) You can tell its not about data types because you can use the same types as the second excerpt, and the operations of the first, and get the same results as the first (which shares operations) and not the second (which shares datatypes):
If you understand how assignment and mutation works in one scope, you understand how argument passing works. Trying to teach a distinction that exists between how different operations affect variables that initially reference the same object as a distinction about how datatypes are passed as arguments is confusing, because you as a teacher are presenting the source of the difference in behavior as originating in a completely different place than where it actually comes from. That's not a problem with Python being complex, it is a problem with you taking a very simple thing and making it complex by ascribing it to a source that is completely irrelevant to what is actually going on.> you dont get too far into python -- especially if you are reading (or copypastaing) other People's code -- before you see if __name__ == "__main__"
First off, if you are teaching someone, you are showing that person the code, and not allowing copy-and-paste.
Second, no, that comes up much less often than you'd expect.
Third, it's the same as `if value == 'example':`. Underscores are not jargon.
Fourth, it's trivially searchable. That's the part where you can copy and paste - into a search engine, which will immediately find you several competent explanations such as https://stackoverflow.com/questions/419163 .
> even "def" is kind of a weird fucking keyword.
Admittedly a poor choice, but not a deal breaker. You need the concept of functions to do programming. But you don't need the concept of data hiding, nor do you need any of the multiple, barely-related concepts described by the term "static".
> Don't get me started about teaching beginners which datatypes are pass by reference and which are pass by value.
There's nothing to explain. They are all pass by value, per the strict meaning of those terms.
Those terms have been widely seen as less than ideal for decades, however, because they fail to account for variables with reference semantics (i.e., what Python uses - which are sometimes called "names"). A more modern term is "pass by assignment", which correctly describes all variable passing in Python: passing an argument to a parameter is a form of assignment, and works the same way as assigning a value to a name.
This is far less complex than C#, in which user-defined types may have either value semantics or reference semantics, and which supports both pass by assignment and two separate, true forms of pass by reference (for initialization and for modifying an existing object). And certainly it's less complex than whatever C++ is doing (see e.g. https://langdev.stackexchange.com/questions/3798 ).
> try explaining to an elementary school student why
First: if someone gives you a bag with three apples in it, you can put another apple in the bag and give it back, and the other person will have a bag with four apples in it. But if you add 3 + 1, that doesn't change the meaning of 3. These are simple ideas that an elementary school student already understands.
Second: from extensive experience teaching beginners (never mind that you are moving the goalposts now), it makes no sense to get into the theory. It's not helpful. A student who can ask about this has already lost the plot, because the two examples use completely different syntax (a method call versus an assignment) so they shouldn't be expected to work similarly. You avoid this problem by using more precise language early on. "Change" is not an appropriate word here.
Third: you give this example because you think that `bar` (and you imply by your naming that a list is being passed) demonstrates pass by reference. This is simply incorrect. Please read https://nedbatchelder.com/text/names1.html.
Fourth: your use of profanity and the overall tone of your writing suggests that you simply don't like the fact that Python works the way that it does. This is not a good look IMO.
Just for the record, I've been in variations of this discussion countless times. I know what I'm talking about. All links above are in my bookmarks.
In my experience people have to first figure out what the hell numpy is and how to get it (venv, conda, pip, uv, uvx, …) because python arrays are shit, and so people fix that wart with an external C library. Then they notice that some other dependency requires a previous python version, but their python is installed globally and other dependencies were installed for that. These are uniquely python-specific problems. Lisp doesn’t have those problems
> what the hell numpy is
Did they try using a search engine? But more to the point, if they don't understand what it is, how did they find out it exists?
> how to get it (venv, conda, pip, uv, uvx, …)
uvx is a command from the same program as uv; venv is not a way to obtain packages; and the choice here isn't a real stumbling block.
> because python arrays are shit
I can't say I've seen many people complain about the standard library `array` module; indeed it doesn't seem like many people are aware it exists in the first place.
If you're talking about lists then they serve a completely different purpose. But your use of profanity suggests to me that you don't have any actual concrete criticism here.
> Then they notice that some other dependency requires a previous python version
Where did this other dependency come from in the first place? How did they get here from a starting point of dissatisfaction with the Python standard library?
> but their python is installed globally and other dependencies were installed for that.
... and that's where actual environment management comes in, yes. Sometimes you have to do that. But this has nothing to do with teaching Python. You have necessarily learned quite a bit by the time this is a real concern, and if you were taught properly then you can self-study everything else you need.
> These are uniquely python-specific problems.
No other languages ever require environment management?
> Lisp doesn’t have those problems
Please tell me about your experience using Qi.
You don’t need to teach it to a beginner. The first of learning doesn’t need more than the standard library and when you need more than that you’re either giving them the single command necessary to run or, more likely, having them use a template project where a tool like Poetry is doing that automatically.
What this usually hits isn’t that managing Python packages is hard in 2025 but that many people do not learn how their operating system works conceptually until the first time they learn to program and it’s easy to conflate that with the first language you learn.
> You don’t need to teach it to a beginner.
gp's claim was:
> Python is better because it's easy for people to learn,
i believe then we agree: it is not.
Let me introduce you to uv[0]. And yes it does say something that this tool isn't written in Python, but I'd say there's even more to be said that so many are trying to support Python.
[0] https://docs.astral.sh/uv/
yeah the reason why that's not an answer is because another half of users will say use poetry. if you want to do bioinformatics, people will insist on conda. then your team will say "use rye" and these strategies are somewhat compatible but ultimately mutually incompatible in ways that will drive you mad every time you hit a tiny snag that nonetheless grinds your attempt to just fucking run code to a halt.
It has become successful largely because it has always had really good foreign function interface. If you have a scientific or mathematical library laying around in C, then you could wire it up to Python, and then suddenly you have all the flexibility of a (fairly clean) scripting language to orchestrate your high speed C.
Good examples of this are numpy and tensorflow.
tensorflow is atrocious, which is why it's basically dead in favor of (py)torch.
someone learning python as their first language knows so little its perfectly fine to let them pollute their global environment. Someone who knows other languages can understand what venv is for.
Instead they can type python to open a shell and use python to immediately run their file.
> perfectly fine
It's not, because you can fuck up system/unrelated app python dependencies and in extreme cases have to reinstall OS. Thankfully as developers migrate away from python/adopt strategies like flatpak this is less of a problem.
Other PLs do not have this problem.
momentum + ecosystem often play a much larger role than actual language merits.
And yet that momentum and ecosystem wouldn't have been achieved in the first place if there weren't enough merits in the language to trigger and maintain that interest.
I think the take should be a bit more nuanced.
Some languages definitely had people gravitate towards them due to being innovative in a given space, but in many of those cases, the comparative advantage was lost to other languages/techs/frameworks that simply failed to gain a market share "equal to their innovative contribution" due to the first comer's advantage.
I think you're being too unfair. People aren't dumb.
It's also about how much better.
Beyond a decent enough type system, the advantages start to flatten and other factors start to matter more.
Can't speak too much for python, but as someone who's written large amounts of code in OCaml and Typescript, the strictest compiler options for Typescript are good enough.
No, people aren't dumb. They're practical. And so they choose to do what is practical, which in this case is to choose Python. And, to me, that makes it the better language.
Aren't there other benefits to server-side parameter binding besides just SQL-injection safety? For instance, using PG's extended protocol (binary) instead of just raw SQL strings. Caching parameterized prepared statements, etc.
Also:
Is dangerously close to: A single character difference and now you've just made yourself trivially injectible.I don't think this new format specifier is in any way applicable to SQL queries.
Templates are a very different duck type from strings and intentionally don't support __str__(). The SQL tool can provide a `safe_execute(Template)` that throws if passed a string and not a Template. You can imagine future libraries that only support Template and drop all functions that accept strings as truly safe query libraries.
> Caching parameterized prepared statements, etc.
Templates give you all the data you need to also build things like cacheable parameterized prepared statements. For DB engines that support named parameters you can even get the interpolation expression to auto-name parameters (get the string "name" from your example as the name of the variable filling the slot) for additional debugging/sometimes caching benefits.
But t"..." and f"..." have different types; we can make db.execute reject character strings and take only template objects.
Yeah that would be a backward compatible way to do stuff.
You solve that with an execute(stmt) function that requires you to pass in a template.
In Javascript, sql`where id = ${id}` is dangerously close to normal string interpolation `where id = ${id}`, and db libs that offer a sql tag have query(stmt) fns that reject strings.
> A single character difference and now you've just made yourself trivially injectible.
No; a single character difference and now you get a `TypeError`, which hopefully the library has made more informative by predicting this common misuse pattern.
> Aren't there other benefits to server-side parameter binding besides just SQL-injection safety? For instance, using PG's extended protocol (binary) instead of just raw SQL strings. Caching parameterized prepared statements, etc.
All of which can be implemented on top of template strings.
> A single character difference and now you've just made yourself trivially injectible.
It's not just a one character difference, it's a different type. So `db.execute` can reject strings both statically and dynamically.
> I don't think
Definitely true.
> this new format specifier is in any way applicable to SQL queries.
It's literally one of PEP 750's motivations.
> > I don't think
> Definitely true.
The rest of your comment is valuable, but this is just mean-spirited and unnecessary.
That would be in addition to doing any runtime checks.
The first mistake we're going to see a library developer make is:
Maybe because they want their execute function to be backwards compatible, or just because they really do want to allow either raw strings are a template string.> they really do want to allow either raw strings are a template string.
I’d consider that an invalid use case:
1. You can create a template string without placeholders.
2. Even if the caller does need to pass in a string (because they’re executing from a file, or t-strings don’t support e.g. facetting) then they can just… wrap the string in a template explicitly.
nitpicking:
> It's not just a one character difference, it's a different type. So `db.execute` can reject strings both statically and dynamically.
in this case, that's not actually helpful because SQL statements don't need to have parameters, so db.execute will always need to accept a string.
You can just pass it a template with no substitutions.
> db.execute will always need to accept a string.
No. A t-string with no placeholders is perfectly fine. You can use that even if you have no parameters.
> Caching parameterized prepared statements, etc.
I didn’t explicitly mention this in my post but, yes, the Template type is designed with caching in mind. In particular, the .strings tuple is likely to be useful as a cache key in many cases.
>> I don't think >Definitely true.
I thought we left middle-school playground tactics behind.
> It's literally one of PEP 750's motivations.
Python is notorious for misguided motivations. We're not "appealing to authority" here. We're free to point out when things are goofy.
> I don't think this new format specifier is in any way applicable to SQL queries.
Agree. And the mere presence of such a feature will trigger endless foot-gunning across the Python database ecosystem.
Quite easy to detect with a proper linter.
t vs f going to be hard to spot.
This is true of many other things, which is why we have type checkers and linters to be perfectly rigorous rather than expecting humans to never make mistakes.
and syntax highlighting
Dang! Thanks for pointing this out.
I had to look SEVERAL times at your comment before I noticed one is an F and the other is a T.
This won’t end well. Although I like it conceptually, this few pixel difference in a letter is going to cause major problems down the road.
How? tstrings and fstrings are literals for completely different types.
CS has survived for decades with 1 and 1.0 being completely different types.
I had an extended debugging session last week that centered on 1 and 1. confusion in a library I have to use...
Yeah, it's a real bummer when that happens. I wish JSON never tried to do types.
Reread my comment. It’s about noticing you have an “f” or a “t” and both are very similar characters.
Yes, but you will get an error since string and templates are different types and have different interfaces.
Click "parent" a few times and look at the code example that started this thread. It's using the same function in a way that can't distinguish whether the user intentionally used a string (including an f-string) and a t-string.
Yes, and the parent is misguided. As was pointed out in multiple replies, the library can distinguish whether an ordinary string or a t-string is passed because the t-string is not a string instance, but instead creates a separate library type. A user who mistakenly uses an f prefix instead of a t prefix will, with a properly designed library, encounter a `TypeError` at runtime (or a warning earlier, given type annotations and a checker), not SQL injection.
In this particular instance it can't, because there are 3 ways in question here, and it can't distinguish between correct intentional usage and accidental usage of an f-string instead of a t-string:
The first and second look identical to execute() because all it sees is a string. But the second one is wrong, a hard-to-see typo of the third.If f-strings didn't exist there'd be no issue because it could distinguish by type as you say. But we have an incorrect SQL-injection-prone usage here that can't be distinguished by type from the correct plain string usage.
There is no reason to support the first or second usage. It's totally fine to always require a t-string:
See? No reason to accept strings, it's absolutely fine to always error if a string is passed.My (and their) point is that's the already existing API. You're proposing a big breaking change, with how many frameworks and tutorials are built on top of that.
It's not like this is the first time APIs have been improved. There are many tools (e.g. deprecation warnings & hints in editors, linter rules) that can help bridge the gap - even if t-strings are only used for new or refactored code, it's still a big improvement!
There's also simply no hard requirement to overload an `execute` function. We have options beyond "no templates at all" and "execute takes templates and strings", for example by introducing a separate function. Why does perfect have to be the enemy of good here?
Because they're both passed to "execute", which can't tell between the f-string and a non-interpolated query, so it just has to trust you did the right thing. Typoing the "t" as an "f" introduces SQL injection that's hard to spot.
Assuming `execute` takes both. You could have `execute(template)` and `execute_interpolated(str, ...args)` but yeah if it takes both you'll have challenges discouraging plain-text interpolation.
It would have to be the other way around or be a (possibly major) breaking change. Just execute() with strings is already standard python that all the frameworks build on top of, not to mention tutorials:
https://docs.python.org/3/library/sqlite3.html
https://www.psycopg.org/docs/cursor.html
https://dev.mysql.com/doc/connector-python/en/connector-pyth...
> It would have to be the other way around or be a (possibly major) breaking change.
If it is going to reject the currently-accepted unsafe usage, its going to be a major breaking change in any case, so I don't see the problem. I mean, if you are lamenting it can't reject the currently-accepted SQL-interpolated-via-f-string because it can't distinguish it by type from plain strings with no interpolation, you are already saying that you want a major breaking change but are upset because the particular implementation you want is not possible. So you can't turn around and dismiss an alternative solution because it would be a major breaking change, that's what was asked for!
`execute` can tell the difference, because `t"..."` does not create the same type of object that `f"..."` does.
Or you could use this in a library like sh with
With t-strings you could run proper escaping over the contents of `some_file` before passing it to a shell.I'd have to take a look at the order things happen in shell, but you might even be able to increase security/foot-gun-potential a little bit here by turning this into something like `stat "$( base64 -d [base64 encoded content of some_file] )"`.
You should check out PEP 787
Oh! I missed this one because I've been looking specifically at the Packaging forum rather than the PEPs forum. This looks like a brilliant use case. (I'm aiming for wide compatibility - back to 3.6 - with my current projects, but I look forward to trying this out if and when it's accepted and implemented.)
Now if only the overall `subprocess` interface weren't so complex....
We really should just point most of these comments at that PEP. Thanks for getting it out so fast.
PEP 787 – Safer subprocess usage using t-strings https://peps.python.org/pep-0787/
Hmm, PEP-787 has some interesting discussions around it. I'll have to sort my thoughts on these aspects a bit.
Not Python but this is exactly the idea behind zx
https://github.com/google/zx
A potential concern is how close this looks to the pattern they're trying to override.
versusThe key point is that t-strings are not strings. Db.execute(t”…”) would throw an exception, because t”…” is not a string and cannot be interpreted as one.
In order for a library to accept t-strings, they need to make a new function. Or else change the behavior and method signature of an old function, which I guess they could do but any sanely designed library doesn’t do.
Handling t-strings will require new functions to be added to libraries.
yes but the bug is writing f instead of t and I assume f will just work.
To clarify even more:
The problem is not writing by mistake t instead of f => this is what we want and then for this we implement a new function
The problem is writing f instead of t => and this will silently work I assume (not a Python dev just trying to understand the language design)
> The problem is writing f instead of t => and this will silently work I assume (not a Python dev just trying to understand the language design)
In the fullness of time it has no reason to. Even in the worst case scenario where you have to compose the query dynamically in a way t-strings can’t support, you can just instantiate a Template object explicitely.
>yes but the bug is writing f instead of t and I assume f will just work
but it will not. f-strings and t-strings are not compatible types, they will not "just work". not unless somebody changes a library to make it just work. as long as nobody does that, it's not an issue.
But won't the f string version fail loudly because there's no name parameter?
the {name} parameter is in the locals() dict like it always is
Good point. Perhaps the database api could refuse strings and require Templates.
That’s a big breaking change around a brand new feature. I’m sure it could be done well, but it gives me the shivers.
You add a new API that takes templates only leaving the existing API in place. You (some releases later) deprecate the string API. You (some releases later, with clear advance warning of when it is coming) actually remove the deprecated API. "It's a big breaking change around a brand new feature", yeah, so you don't break anything around a brand new feature, it's not like this kind of transition is a new concept.
much better would be execute_template(t"...")
I don't see what it adds over f-string in that example?
The execute function can recognize it as a t-string and prevent SQL injection if the name is coming from user input. f-strings immediately evaluate to a string, whereas t-strings evaluate to a template object which requires further processing to turn it into a string.
Then the useful part is the extra execute function you have to write (it's not just a substitute like in the comment) and an extra function can confirm the safety of a value going into a f-string just as well.
I get the general case, but even then it seems like an implicit anti-pattern over doing db.execute(f"QUERY WHERE name = {safe(name)}")
Problem with that example is where do you get `safe`? Passing a template into `db.execute` lets the `db` instance handle safety specifically for the backend it's connected to. Otherwise, you'd need to create a `safe` function with a db connection to properly sanitize a string.
And further, if `safe` just returns a string, you still lose out on the ability for `db.execute` to pass the parameter a different way -- you've lost the information that a variable is being interpolated into the string.
db.safe same as the new db.execute with safety checks in it you create for the t-string but yes I can see some benefits (though I'm still not a fan for my own codebases so far) with using the values further or more complex cases than this.
Yeah but it would have to be something like `db.safe("SELECT * FROM table WHERE id = {}", row_id)` instead of `db.execute(t"SELECT * FROM table WHERE id = {row_id}")`.
I'd prefer the second, myself.
No, just `db.execute(f"QUERY WHERE name = {db.safe(name)}")`
And you add the safety inside db.safe explicitly instead of implicitly in db.execute.
If you want to be fancy you can also assign name to db.foos inside db.safe to use it later (even in execute).
This is just extra boilerplate though, for what purpose?.
I think one thing you might be missing is that in the t-string version, `db.execute` is not taking a string; a t-string resolves to an object of a particular type. So it is doing your `db.safe` operation, but automatically.
Of course you can write code like that. This is about making it easier not to accidentally cause code injection by forgetting the call of safe(). JavaScript had the same feature and some SQL libraries allow only the passing of template strings, not normal strings, so you can't generate a string with code injection. If you have to dynamically generate queries they allow that a parameter is another template string and then those are merged correctly. It's about reducing the likelihood of making mistakes with fewer key strokes. We could all just write untyped assembly instead and could do it safely by paying really good attention.
But if someone omits the `safe` it may still work but allow injection.
Same is true if someone forgets to use t" and uses f" instead.
At least db.safe says what it does, unlike t".
Your linter can flag the type mismatch, and/or the function can reject f"" at runtime. This is because t"" yields a Template, not a str.
Template is also more powerful/concise in that the stringify function can handle the "formatting" args however it looks.
Note also, that there's no requirement that the template ever become a str to be used.
Not really, since f"" is a string and t"" is a template, you could make `db.execute` only accept templates, maybe have
`db.execute(Template)` and `db.unsafeExecute(str)`
agreed. but then you're breaking the existing `db.execute(str)`. if you don't do that, and instead add `db.safe_execute(tpl: Template)`, then you're back to the risk that a user can forget to call the safe function.
also, you're trusting that the library implementer raises a runtime exception if a string a passed where a template is expected. it's not enough to rely on type-checks/linting. and there is probably going to be a temptation to accept `db.execute(sql: Union[str, Template])` because this is non-breaking, and sql without params doesn't need to be templated - so it's breaking some stuff that doesn't need to be broken.
i'm not saying templates aren't a good step forward, just that they're also susceptible to the same problems we have now if not used correctly.
Then make `db.unsafe_execute` take a string.
Yeah, you could. I'm just saying that by doing this you're breaking `db.execute` by not allowing it to take it string like it does now. Libraries may not want to add a breaking change for this.
What does db.safe do though? How does it know what is the safe way of escaping at that point of the SQL? It will have no idea whether it’s going inside a string, if it’s in a field name position, denotes a value or a table name.
To illustrate the question further, consider a similar html.safe: f"<a href={html.safe(url)}>{html.safe(desc)</a>" - the two calls to html.safe require completely different escaping, how does it know which to apply?
The first one already exists like:
But you have to remember to call the right safe() function every time:
Oops, you're screwed and there is nothing that can detect that. No such issue with a t-string, it cannot be misused.Some SQL engines support accepting parameters separately so that values get bound to the query once the abstract syntax tree is already built, which is way safer than string escapes shenanigans.
I’d always prefer to use a prepared statement if I can, but sadly that’s also less feasible in the fancy new serverless execution environments where the DB adapter often can’t support them.
For me it just makes it easier to identify as safe, because it might not be obvious at a glance that an interpolated template string is properly sanitised.
> and an extra function can confirm the safety of a value going into a f-string just as well.
Yes, you could require consumers to explicitly sanitize each parameter before it goes into the f-string, or, because it has the structure of what is fixed and what is parameters, it can do all of that for all parameters when it gets a t-string.
The latter is far more reliable, and you can't do it with an f-string because an f-string after creation is just a static string with no information about construction.
> Then the useful part is the extra execute function you have to write
Well, no, the library author writes it. And the library author also gets to detect whether you pass a Template instance as expected, or (erroneously) a string created by whatever formatting method you choose. Having to use `safe(name)` within the f-string loses type information, and risks a greater variety of errors.
If I pass an f-string to a method, it just sees a string. If I pass a t-string the method can decide how to process the t-string.
Wouldn't this precisely lead to sql injection vulnerabilities with f-strings here?
f-strings won’t sanitize the value, so it’s not safe. The article talks about this.
The article talked about it but the example here just assumes they'll be there.
What do you mean by "they"? You mean the template interpolation functions?
Yes, the idea is that by having this in the language, library authors will write these implementations for use cases where they are appropriate.
The sanitization. Just using a t-string in your old db.execute doesn't imply anything safer is going on than before.
Your "old" db.execute (which presumably accepts a regular old string) would not accept a t-string, because it's not a string. In the original example, it's a new db.execute.
Using a t-string in a db.execute which is not compatible with t-strings will result in an error.
Using a t-string in a db-execute which is, should be as safe as using external parameters. And using a non-t-string in that context should (eventually) be rejected.
Again, just because a function accepts a t string it doesn't mean there's sanitization going on by default.
Yes, but if a function accepts a template (which is a different type of object from a string!), either it is doing sanitization, or it explicitly implemented template support without doing sanitization—hard to do by accident!
The key point here is that a "t-string" isn't a string at all, it's a new kind of literal that's reusing string syntax to create Template objects. That's what makes this new feature fundamentally different from f-strings. Since it's a new type of object, libraries that accept strings will either have to handle it explicitly or raise a TypeError at runtime.
I'm not sure why you think it's harder to use them without sanitization - there is nothing inherent about checking the value in it, it's just a nice use.
You might have implemented the t-string to save the value or log it better or something and not even have thought to check or escape anything and definitely not everything (just how people forget to do that elsewhere).
I really think you're misunderstanding the feature. If a method has a signature like:
It would be weird for the implementation to just concatenate everything in the template together into a string without doing any processing of the template parameters. If you wanted an unprocessed string, you would just have the parameter be a string.I'm not. Again, you might be processing the variable for logging or saving or passing elsewhere as well or many other reasons unrelated to sanitization.
Taking a Template parameter into a database library's `execute` method is a big bright billboard level hint that the method is going to process the template parameters with the intent to make the query safe. The documentation will also describe the behavior.
You're right that the authors of such libraries could choose to do something different with the template parameter. But none of them will, for normal interface design reasons.
A library author could also write an implementation of a `plus` function on a numerical type that takes another numerical type, and return a string with the two numbers concatenated, rather than adding them together.
But nobody will do that, because libraries with extremely surprising behavior like that won't get used by anybody, and library authors don't want to write useless libraries. This is the same.
The original comment said that it'd replace
with It's true that in theory `db.execute` could ignore semantics and concatenate together the template and variables to make a string without doing any sanitisation, but isn't the same true of the syntax it was claimed to replace?Just because templates (or the previous syntax of passing in variables separately) could be used in a way that's equivalent safety-wise to an f-string by a poorly designed library does not mean that they add nothing over an f-string in general - they move the interpolation into db.execute where it can do its own sanitization and, realistically, sqlite3 and other libraries explicitly updated to take these will use it to do proper sanitization.
Sure, and the safe() function proposed upthread might also just be doing logging.
Because t-strings don't create strings, so if the library doesn't support t-strings the call can just error.
it makes it so people too lazy to make good types and class will be getting closer to sane code without doing sane code...
imagine writing a SqL where u put user input into query string directly.
now remember its 2025, lie down try not to cry.
safety against sql injection
3. It prevents the developer from trying
or or other ways of manually interpolating the string - because `db.execute` can flag a `TypeError` if given a string (no matter how it was constructed) rather than a `Template` instance.Python is not the first one to get this feature. It's been present in JS for some time now, and before that in C# (not sure if that's the origin or they also borrowed it from somewhere). Python adopted it based in part on successful experience in those other languages.
That's really cool. I don't use JS or C#, so I wasn't aware of this, but it's a good idea.
Assuming you also need to format non-values in the SQL (e.g. column names), how does the `execute` function is supposed to make the difference between stuff that should be formatted in the string vs a parametrized value?
Same as currently: the library provides some sort of `Identifier` wrapper you can apply to those.
Fair enough. It would be nice if Python allowed to customize the formatting options after `:`
This way you could encode such identifier directly in the t-string variable rather than with some "out-of-band" logic.
> Fair enough. It would be nice if Python allowed to customize the formatting options after `:`
It does, the `Interpolation` object contains an arbitrary `format_spec` string: https://peps.python.org/pep-0750/#the-interpolation-type
However I think using the format spec that way would be dubious and risky, because it makes the sink responsible for whitelisting values, and that means any processing between the source and sink becomes a major risk. It's the same issue as HTML templates providing `raw` output, now you have to know to audit any modification to the upstream values which end there, which is a lot harder to do than when "raw markup" values are reified.
> rather than with some "out-of-band" logic.
It's the opposite, moving it to the format spec is out of band because it's not attached to values, it just says "whatever value is here is safe", which is generally not true.
Unless you use the format spec as a way to signal that a term should use identifier escaping rules rather than value escaping rules (something only the sink knows), and an `Identifier` wrapper remains a way to bypass that.
> Unless you use the format spec as a way to signal that a term should use identifier escaping rules rather than value escaping rules (something only the sink knows)
This should be quiet common in the SQL applications. It will be nice to write t"select {name:id} from {table:id} where age={age}" and be confident that the SQL will be formatted correctly, with interpolations defaulting to (safe) literal values.
The article does mention that the function receiving the template has access to those formatting options for each interpolation, so presumably you could abuse the ones that are available for that purpose?
One thing it misses is compile-time checks for e.g. the format spec.
Doesn't all of Python miss that, having (close to) no compile time?
Python does some checks before it runs code. E.g.:
gives: before printing hello, and note that f() wasn't even called.I think it's just giving an error because a valid AST can't be made, which means valid bytecode can't be made. "<word> <word>" is only valid syntax if one is a reserved word. `nonlocal(foo)` is just fine, of course.
No, it gives an error because `nonlocal foo` requests that the name `foo` be looked up in a closure, but `f` doesn't have such a closure (the `foo` defined outside the function is global instead). `nonlocal` is the same sort of keyword as `global` but for enclosing functions instead of the global namespace; see also https://stackoverflow.com/questions/1261875 .
Here's the statement checking code, which I believe is pre-AST [1]. I would have to dig more to see if that check is there to prevent invalid AST or to just "help the user" (would depend on how they reference the original variable I suppose).
But wow, that's the first time I've seen "nonlocal". In the ~100 packages I have installed, I see 0 usages!
[1] https://github.com/python/cpython/blob/a6a3dbb7db0516a72c5ef...
Well, yes, not a lot of people write closures except perhaps when they implement decorators. So there's ordinarily no non-local scope to worry about. People tend to write classes instead, because that's what's familiar.
> "<word> <word>" is only valid syntax if one is a reserved word.
`nonlocal` is a keyword
Do t-strings miss something that f-strings provides for format_spec etc.?
FWIW, format_spec is available in the template structure, so the function writer could at least do a runtime check.
> Allowing library developers to do whatever they want with {} expansions is a good thing, and will probably spawn some good uses.
I completely disagree with this. Look what happened to Log4J when it was given similar freedoms.
I think this would have solved the log4j vulnerability, no?
As I understand it, log4j allowed malicious ${} expansion in any string passed to logging functions. So logging user generated code at all would be a security hole.
But Python's t-strings purposely _do not_ expand user code, they only expand the string literal.
Yes and your example is the hero case because it isn't just sugar. A t-string implementation for SQL will of course escape the values which is a common security issue.
https://xkcd.com/327/
No, a t-string returns a Template which is basically { strings: str[], values: any[] }.
So you would write db.execute(template) to turn template t"... where id = {id}" into a parameterized structure like ("... where id = ?", id).
Now instead of being explicit all it takes is someone unfamiliar with t strings (which will be almost everyone - still few know about f strings and their formatting capabilities) to use an f instead and you are in for a bad time.
Any sane library will just error when you pass a string to a function that expects a template though. And that library will have types too so your IDE tells you before you get that far.
Such library functions tend to also accept a string as a valid input. E.g. db.execute from the GP usually works with strings to allow non-parametrized SQL queries.
> Such library functions tend to also accept a string as a valid input.
Also? They tend only to accept a string (possibly with some additional arguments, if there is an in-library way to handle parameterization) as input, because Template literally hasn't been an option. New APIs designed with Template available will look different.
The library should just refuse strings. If a non parametrized query is desired, it could require the user to supply a t-string with no {}.
This would break backwardcompatibility pretty hard. In many cases it may not be worth it.
Javascript already has prior art here.
A library can extend an existing database library like 'pg' so that PgClient#query() and PgPool#query() require string template statements.
That way 'pg' can continue working with strings, and people who want nice templated strings can use the small extension library, and the small extension library makes it impossible to accidentally pass strings into the query functions.
But now at least the language has the necessary rope (and an opportunity for a cultural push to insist on it.)
That is an issue, but essentially it boils down to the existing risk of unknowledgeable people not escaping untrusted inputs. The solution should be more education and better tooling (linters, SAST), and t-strings are likely to help with both.
t-strings allow building APIs which don't accept strings at all (or require some sort of opt-in), and will always error on such. That's the boon.
Having to write
even when there's nothing to format in is not a big imposition.I suppose lack of overlap in the "interface surface" (attributes, including callables) between `str` and `Template` should nip the kind of issue in the bud -- being passed a `Template` and needing to actually "instantiate" it -- accessing `strings` and `values` attributes on the passed object, will likely fail at runtime when attempted on a string someone passed instead (e.g. confusing a `t`-string with an `f`-string)?
No, because they don't return a string, so good library authors will raise a type error when that happens, for exactly this reason.
> In addition, I hope that the tooling ecosystem will adapt to support t-strings. For instance, I’d love to see black and ruff format t-string contents, and vscode color those contents, if they’re a common type like HTML or SQL.
This is such a strange take on t-strings. The only way for anything to infer that the template string is supposed to turn into valid HTML or SQL is to base it of the apparent syntax in the string, which can only be done in an ad-hoc fashion and has nothing to do with the template string feature.
The way the feature has been designed there is no indication in the string itself what type of content it is or what it will eventually be converted to. It’s all handled by the converting function.
As others have added, something like sql”select * from {table}” would have been able to do this, but there’s not even any guarantees that something that is in a template that will be converted into valid sql by a converting function should be any type of valid sql prior to that conversion. For all you know t“give me {table} but only {columns}” might be a converted into valid sql after the template is processed.
> This is such a strange take on t-strings
I understand why this seems strange at first!
As Paul mentioned, we spent quite a lot of time considering these issues as PEP 750 came together. In the end, we concluded (a) the PEP leaves open quite a few potential approaches for tools to adopt (not just the one you suggest, as others here have pointed out), and (b) it's ultimately something that the broader tooling community needs to rally around and should probably be out of scope for the PEP itself.
So, with that background in mind, I am indeed hopeful we'll see the ecosystem adapt! :-)
I get that. But that “ecosystem might adapt” has absolutely nothing to do with t-strings is what I’m saying. Anyone who wants to validate sql will have to just look through strings guess that it’s sql and validate and highlight it. This could be done before and after just the same. The template strings by nature of just being “strings that will turn into something else at some later point” do nothing to further this in any way.
I think you'll just see a pattern like:
And highlighters and static analyzers will key off of this.JavaScript's tagged template literals are actually about as flexible as this, since you can dynamically choose the tag function, it's just very rare to do so, so tools assume a lot based on the name of the function. Python tools can basically do the same thing, and just not support t-strings that aren't nested inside a well-named processing function.
> And highlighters and static analyzers will key off of this.
Then the t is redundant and we don’t need t strings involved in any way. This would work just as well. In fact better by using html("<h1>Hello</h1>") and have it return a html object instead of returning a just a string which is what t-strings will do. So literally the only contribution from t-strings in that context is to make things worse.
If they had just standardized some way of putting semantics into the template everything would be better. Let us define a “sql = temlate(….)” and use sql”…”
Another option would be type hints, something like `title: HTMLTemplate = t"<h1>Hello</h1>"`.
The original PEP and the original discussion had this in scope. We removed it to let this emerge later. There are different ways to signal the language -- some more friendly to tooling, some more robust.
PyCharm (as well as the other JetBrains IDEs) has for at least 10 years supported language injection in Python strings using a comment [0]. Worst case scenario there's no reason whatsoever that that couldn't be used by formatters to apply auto-formatting in a structured and deterministic way.
But that's far from the only option, either! IntelliJ for Java also supports annotations on arguments [1] that then mean that you get syntax highlighting everywhere you use a string literal with the given function:
In Python, typing.Annotated appears to have been specifically designed for purposes like this [2]:> If a library or tool encounters an annotation Annotated[T, x] and has no special logic for the metadata, it should ignore the metadata and simply treat the annotation as T. As such, Annotated can be useful for code that wants to use annotations for purposes outside Python’s static typing system.
So something like this should be perfectly viable:
Now, where you're right is that we shouldn't actually require template strings to accomplish this! As noted, JetBrains has been doing this since forever. But maybe template strings will be useful enough for purposes like this that the tooling will actually evolve to support it in formatters and other editors (and maybe PyCharm can get some of the better support that Java has from JetBrains).[0] https://www.jetbrains.com/help/pycharm/using-language-inject...
[1] https://www.jetbrains.com/help/idea/using-language-injection...
[2] https://docs.python.org/3/library/typing.html#typing.Annotat...
I'm one of the PEP authors and also with JetBrains. I'm talking to the PyCharm team about this.
Couldn’t you do this with a type annotation? e.g. SQLAlchemy could have a SQL type so tools like mypy could see a Template instance and confirm you’re using it safely but Black, Ruff, or SQLFluff could look for the more specialized Annotated[Template, SQL] to realize that the template could be formatted as SQL, and something like Django could even have Annotated[Template, Email], Annotated[Template, HTML], or Annotated[Template, JSX] to indicate what context the same templating syntax is targeting.
This is what we discussed in the first revision of the PEP (the use of `Annotated`.) But we found out: linters don't know anything about the Python type system.
We hope to get a community around all of this, stuff at PyCon US, EuroPython, etc. and work some of this out. The JSX/TSX world really has good tooling. We can provide that for those that want it, perhaps better on some aspects.
Interesting, thanks for the background. I’ve been curious what Astral is going to do in the space but also worry about what happens when their funding runs out.
> The only way for anything to infer that the template string is supposed to turn into valid HTML or SQL is to base it of the apparent syntax in the string
Not the only thing. You can also look at how it is used. Your editor could know of how some popular libraries use t-strings, track which t-strings get passed into functions from those libraries, and use that to assume what grammar the t-string should follow.
Is that cheating? In some sense, yes, but it also is useful and likely will be worth it for quite a few programmers.
Python has had type annotations for a decade, and modern IDEs can interpret them.
Writing `query: SQL = t"SELECT ..."` is a small price to pay for such a DX boost.
Will this allow neat SQL syntax like the following?
If the db.get() function accepts a template, it should, right?This would be the nicest way to use SQL I have seen yet.
This definitely seems like one of the motivations for implementing this feature in the first place - https://peps.python.org/pep-0750/#motivation.
Having more control over the interpolation of string values is a win IMO.
You would need triple-quotes to span multiple lines, but yes, that is exactly how it's intended to work. `db.get` will receive a Template instance, which stores string parts something like `('\n SELECT * FROM users\n WHERE city=', ' AND age>', '\n')` and interpolated parts like `(Interpolation('London'), Interpolation(21))`. It's then responsible for assembling and executing the query from that.
The SQLite extension for Tcl offers something similar:
https://sqlite.org/tclsqlite.html#the_eval_methodAs I understand, that's less powerful, because you can do:
but you can't do: Instead, you have to write: There's no reason you couldn't have such power in Tcl, though: it's just that the authors of SQLite didn't.That’s the sort of thing people have built with the equivalent feature in JavaScript, so it should do. Eg https://github.com/andywer/squid is a nice example.
Yes, it's quite delightful.
Isn't the actually proper way to use prepared statements anyway? If we are doing that properly, then what does this t string business buy us for SQL usage from Python?
Because, as the article states, people aren’t using prepared statements. Instead, they pass f-strings because they’re more convenient.
Except to maintain backwards compatibility we're probably going to get new methods that only accept templates, completely circumventing any effort to stop people passing in strings.
Prepared statements were the recommended way to run SQL queries when I was starting with PHP 15 years ago, anyone writing code vulnerable to SQL injection at this point should not be writing code.
Well yes but the alternative is to never make language improvements because legacy code exists.
f strings are syntax rather than a type, the resulting templates look like a reasonable way to specify a prepared statement.
> This would be the nicest way to use SQL I have seen yet.
EF/EF Core has existed for years :)
https://learn.microsoft.com/en-us/ef/core/querying/sql-queri...
I've used it for years. In order to generate the models you had to use the visual designer which was slow and buggy.
Generally annoying experience if you have to clock in and out every day to watch that UI break your database relations whenever you click save.
No one uses it today, or in the last 5 years or so I presume. You use https://learn.microsoft.com/en-us/ef/core/modeling/#use-data...
This was a completely separate, legacy extension of VS, not EF let alone EF Core.
Completely separate is pushing it since it was recommended by Microsoft, but yes, I am old and times have changed.
Thanks, I hate it. While it's nice syntactic sugar, the difference between an SQL injection vulnerability and a properly parametrized query is now a single letter that's easily missed
The t-string produces a Template object without a __str__() method. You can’t mistakenly use an f-string in its place. Either the code expects a string, in which case passing it a Template would blow it up, or the code expects a Template, in which case passing it a string would blow it up.
> or the code expects a Template, in which case passing it a string would blow it up.
That's where the problem is though -- in most cases it probably won't blow up.
Plenty of SQL queries don't have any parameters at all. You're just getting the number of rows in a table or something. A raw string is perfectly fine.
Will sqlite3 really disallow strings? Will it force you to use templates, even when the template doesn't contain any parameters?
You can argue it should, but that's not being very friendly with inputs, and will break backwards compatibility. Maybe if there's a flag you can set in the module to enable that strict behavior though, with the idea that in a decade it will become the default?
There's definitely a backwards compatibility hurdle of switching to a template-only API, but a template-only API doesn't look that much "less friendly" with inputs, when the only difference is a `t` before every string, regardless of number of parameters.
Sure, but it's just I don't have to do that anywhere else.
I never put an f in front of a string if I'm not putting variables within it.
And I'm generally used to Python inputs being liberal. I can usually pass a list if it expects a tuple; I can pass an int if it expects a float; often I can pass an item directly instead of a tuple with a single item. Regex functions take regular strings or regex strings, they don't force regex strings.
Being forced to use a single specific type of string in all cases is just very different from how Python has traditionally operated.
It's safer, I get that. But it's definitely less friendly, so I'll be curious to see how module maintainers decide to handle this.
> Being forced to use a single specific type of string in all cases is just very different from how Python has traditionally operated.
Maybe that's partly the disconnect here? "t-string" is probably a confusing colloquial name because they aren't strings, they are Templates. The runtime type is a Template. It is a very different duck-type from a string. As a duck-typable object it doesn't even implicitly or explicitly act like a string, there's intentionally no `__str__()` method and `str(someTemplate)` doesn't work like you'd expect. It shouldn't be a surprise that there is also no implicit conversion from a string and you have to use its own literal syntax: it isn't a string type, it's a Template type.
Python here is still liberal with respect to Templates (it is still a duck type). If a function expects a Template and you don't want to use the t"" shorthand syntax nor use the Template constructor in string.templatelib, you just need a simple class of object that has an `__iter__()` of the correct shape and/or has `strings` and `values` tuples.
Sure, it may make sense for some types of APIs to support a Union of str and Template as "liberal" options, but it's a different class of liberal support from Union of list and tuple or Union of int and float which are closer "domains" of types. A Template isn't a string and at runtime looks nothing like one (despite how syntactically it looks like one at "compile time"). Given `__iter__()` in Template, it may make more sense/would be more "natural" to Union Template with List or Tuple more than with a single string.
Er… that’s just not correct? Python can be more liberal but it’s not always. It depends entirely on the tooling. Libraries will take time to catch up but I can definitely see people creating libraries that enforce t-strings, even if they’re deconstructing them under the hood for legacy libraries.
What's not correct? Python inputs usually are liberal. I didn't say always.
Are you claiming it's traditionally common in Python to be strict with inputs, and that being liberal is the exception?
That Python lets you blindly interchange different types for no good reason. It simply doesn’t.
Yes, it’s common for Python to be strict for inputs when the types are different. For example, try:
Decimal(‘3.0’) / 1.5
You’ll get an error and for good reason.
But... it usually does. For example, try:
It works fine. It doesn't work with a float, which is for good reason. That's the whole point -- its general philosophy is to be pretty liberal with types, except when there's a good reason not to be. Heck, you can even do dumb things like: And get 5 back. If that's not "blindly interchanging different types for no good reason" then I don't know what is. You can even multiply your Decimal object by False and get an answer...Or it's like my original example -- the Regex module isn't restricted to r-strings. It happily works with regular strings. Python's general philosophy is to handle input liberally. Even type hinting is an add-on. Now, it doesn't go as far as JavaScript in allowing e.g. "4"+1, but it's still awfully liberal. I just don't see how you can claim otherwise.
I know about Decimal/int mixing but that’s for a good reason: it’s fine to intermix here. But not for floats (precision issues). The bool/int mixing isn’t “good.” It’s a bad implementation detail that Python is stuck maintaining forever. I’m actually stunned that you think to use this as an example when I think I’d fire any programmer that did that in my team for gross negligence.
The reason it works is because Python functionally has no bool type. True and False are just integers with names. It’s stupid and shouldn’t work like that but it does for historic reasons.
Your example of regex makes no sense either. There is no difference between strings and r-strings. They’re literally the same thing to the interpreter, so how could the regex functions enforce you use r-strings? Maybe they should be different but, for historic reasons, they can’t be without Python 4.0.
> The reason it works is because Python functionally has no bool type. True and False are just integers with names.
This has not been true since around 2.4 or 2.5. The oldest Python I have available to me currently is 2.7, and this holds then, as it does now in 3.13:
Prior to having a bool type, Python didn't even have True/False keywords.The reason something silly like `4 + True` works is because the bool type implements `tp_as_number` [0]. The reason it works this way is intentional because it would been a Python 3 str-style debacle if ints and bools were not interchangeable.
[0] https://github.com/python/cpython/blob/main/Objects/boolobje...
> I’m actually stunned that you think to use this as an example when I think I’d fire any programmer that did that in my team for gross negligence.
You seem to be having a different conversation than I am.
I'm just describing Python as it is. I'm not defending it. I know why you can add True to a number, or else I wouldn't have come up with the example. And I know perfectly well that r-strings are just strings. Python easily could have made them a distinct object, to force people from ever making backslash errors, and restricted Regex functions to them, but didn't.
My only point has been, "Pythonic" things tend to be pretty liberal in what they accept. Type hints aren't even enforced, when they exist at all. You seem to think it shouldn't be that way. Great! But regardless, claiming it's not that way -- that Python is somehow this strict language -- is just mischaracterizing it.
> My only point has been, "Pythonic" things tend to be pretty liberal in what they accept
Being able to use a string as a string and an int as an int are not “pretty liberal in what they accept,” it’s just programming language theory 101! I think you’re mistaking duck typing for “liberal acceptance,” which are not the same thing. There’s always been an expectation that you should use compatible interfaces, even within the standard library. I’ve been bitten enough times by passing a generator in when a function expects a list, for example.
I'm not mistaking it at all. Yes, duck typing is very much liberal acceptance, but Python code tends to go much farther. I could give a million examples -- like how in Python's isinstance() the second argument can be a type or a tuple of types, or in sqlite3 that you can run queries on a connection or on a cursor, and don't even get me started on Matplotlib or Numpy. It's just idiomatic in Python to make things easy for the programmer by accepting multiple types when possible. If you don't recognize this as the common Python pattern, I literally don't know what else to tell you.
> its general philosophy is to be pretty liberal with types, except when there's a good reason not to be.
And there’s a good reason not to be here.
For the reasons discussed above, I'm not sure that it will be the case for t-strings. I think it'll take a little while for frameworks/libraries to adapt (while still maintaining backward compatibility) and a while for best practices to find their way into our linting and other tools.
If you can use a string anywhere you can use a t-string, then a non parametrized t-string is a code smell (lining error). If there is a dedicated template-string API, then there is the implicit threat you are breaking backwards compatibility to stop using regular strings.
> If you can use a string anywhere you can use a t-string
You can't; they're different types. t-strings are not `str`
It's up to good framework/API design to take advantage of this.
A library writer ultimately has to decide if they accept both types. For a database cursor, do you take regular strings + parameter arguments and template strings? Or dedicate a new API to the idea?
If ‘execute’ takes string and t-string, then I would consider it a problem to use a t-string without parameters. If there is a novel API just for t-strings, then you are implying widespread breaking changes as you have a schism between the two ways of providing parameters.My point was that library authors will need to consider this carefully. If you're writing a library where injection attacks matter, then -- long term -- you almost certainly do not want a single method that accepts `Union[str, Template]`. You probably either want to avoid accepting `str` entirely, or perhaps provide two separate methods. Some period of deprecation seems inevitable.
Yeah, "t-string" is possibly a misnomer, because they are in fact at runtime a Template object (from string.templatelib).
Linters complain because f"hello" and "hello" are the _exact_ same string. t"hello" isn't even a string.
And I'm guessing lots of code will expect strings to maintain backward compatibility.
I think it's way more likely that existing libraries will introduce new methods that use t-strings and are type safe, rather than entirely defeat the purpose of having a t-string API.
I'm guessing no existing functions will be extended to allow t-strings for this very reason. Instead, new functions that only accept t-strings will be created.
There's an obvious risk here, same as with strcpy (no, strncpy.. no, strlcpy... no, strcpy_s) that documentation tends to outlive code, and people keep pasting from tutorails and older code so much that the newer alternatives have a hard time cutting through the noise.
I would argue that as bad as some w3schools tutorials were, and copying from bad Stackoverflow answers, going back to MSA and the free cgi archives of the 90s, the tendency of code snippets to live on forever will only be excarbated by AI-style coding agents.
On the other hand, deprecating existing methods is what languages do to die. And for good reason. I don't think there's an easy answer here. But language is also culture, and shared beliefs about code quality can be a middle route between respecting legacy and building new. If static checking is as easy as a directive such as "use strict" and the idea that checking is good spreads, then consesus can slowly evolve while working code keeps working.
It's pretty common for Python libraries to deprecate and remove functionality. It makes people mad, but it's a good thing, for this reason.
Do the python type checkers / linters / whatever have the ability to warn or error on calling certain functions? That would be nice to eventually enforce migration over to the newer functions that only take a t-string template
They sure do, e.g. https://docs.astral.sh/ruff/rules/pandas-use-of-dot-is-null/
Yeah. A while back I was poking through some unfamiliar code and noticed that my editor was rendering a use of `datetime.utcnow()` as struck through. When I hovered it with my mouse, I got a message that that function had been deprecated.
Turns out my editor (vscode) and typechecker (pyright) saw that `datetime.utcnow()` was marked as deprecated (I know one can use the `@deprecated` decorator from Python 3.13 or `__future__` to do this; I think it was done another way in this particular case) and therefore rendered it as struck through.
And it taught me A) that `utcnow()` is deprecated and B) how to mark bits of our internal codebase as deprecated and nudge our developers to use the new, better versions if possible.
Can you do it for functions defined by other people, or only for functions that you defined?
I'm thinking in the general case, but motivated by this example of a 3rd party function that accepts a SQL query as a string, and we'd like everywhere in our codebase to stop using that and instead use the 3rd party function that accepts the query as a t-string
If it's not a completely new library written exclusively around templates, such code currently accepts strings and will most likely continue to accept strings for backwards compatibility.
In that case I don’t understand the security regression that t-strings cause (see GP). Before it was all just strings, but you had to make sure to use them in the correct place. Now you can still just use strings for backwards compat. but you can also move on to a distinctly-typed solution for SQL and the like.
That's entirely implementation dependent. For existing libraries I would expect something like
it would likely be safer to have a safe (accepting Templates) and an unsafe (accepting strings) interface.
Now whether maintainers introduce `getSafe` and keep the old behavior intact, or make a breaking change to turn `get` into `getUnsafe`, we will see
And they could add deprecation warnings gradually
That would be a great argument if Python wasn't a language that let you reach into the internals and define __str__() for things you shouldn't be defining it for. And that is something people will definitely do because, you know, they just need something to friggin work so they can get whatever ticket closed and keep some metric happy tracking time-to-close
Programmers being lazy and shit at their jobs is not a reason to not improve the language.
Type checkers to the rescue ahaha I think db.get could also raise if the type does not match?
I guess that is a misunderstanding on your side, about how templates work. Less hate and more love might help to avoid this type of hotheaded misconception ;-)
Why do you think changing a letter would cause a vulnerability? Which letter do you mean?
f'' vs t'' probably.
Those are two different types
Wow that's only slightly better than using the lowercase letter L vs the digit 1 or letter O vs zero to convey a significant difference.
OP is referring to swapping t with f.
That would result in a string passed to get() and raise an error as get() operates on a template, not on a string.
except if get() can also accept a raw string, which is likely
No sane library is going to do that. If they do let you pass a raw string it should be a different function with the risks clearly documented.
The thing this replaces is every library having their own bespoke API to create a prepared statement on their default/safe path. Now they can just take a template.
How about every library that wants to preserve backwards compatibility?
Or are you suggesting that e.g. every database module needs to implement a new set of query functions with new names that supports templates? Which is probably the correct thing to do, but boy is it going to be ugly...
So now you'll have to remember never to use 'execute()' but always 'execute_t()' or something.
You don't have to remember it, you can use deprecation warnings and lint tools to remind you. (Until eventually the safe API is the only API and then you really have nothing to remember.)
I’d assume their current safe function isn’t taking a string, and is taking some sort of prepared statement? So they could have it take either their prepared statement or a template, and deprecate their prepared statement.
If a library has functions taking a string and executing it as SQL they probably shouldn’t make that take a template instead, but I’d hope that’s a separate explicitly unsafe function already.
For sqlite3, it absolutely takes a regular string.
If you want to substitute parameters, you put a '?' in the string for each one, and provide an additional (optional) tuple parameter with the variables.
So no, there's no explicitly unsafe function. That's my point.
Gotcha. I’d guess they’d want to deprecate that function and create a new one that only accepts a template then, which is definitely annoying! I figured they’d already have more separation between prepared and raw strings which would make it easier.
Why would it?
Also I wonder how easy it will be to shoot oneself in the foot. It may be easy to accidentally make it to a string too soon and not get the proper escapeing.
That’s a library author problem, so it’s less likely since library authors tend to be fewer in number and, for popular libraries, get a reasonable number of eyes on this type of change.
Personally, this feels like a feature that is too focused on one problem to be a general feature. Python is getting huge. When people ask me if Python is easy and simple to learn I have to say "the basics, yes, but to to learn the whole language... not so much".
I feel like in this sense Go really is interesting by rejecting almost every single feature. Honestly not sure generics were worth it as they add a lot of complexity, and while they are nice, I don't need them very much. The general idea to keep the language at its original focus is the right idea IMO. C++ would be the most extreme case where the language itself barely resembles what it started out as.
Python has always been a batteries-included language, so having a go at templated string interpolation —a feature other languages have had for decades— seems like a strange gripe.
It's far more essential than little utilities like textwrap or goliath packages like Python's bundled tkinter implementation.
Batteries included to me also meant a large standard library, not a large language.
What other languages have this feature? I'm not aware of any
C# has InterpolatedStringHandler which isn't quite the same thing, but (in my understanding), trying to address the same core issues.
C# InterpolatedString is very close, with the twisty bit being that C# can rely on static typing for safety so the "f-string" and "t-string" variants use the same literal syntax and depend on what function they are passed to, whereas in both Python and Javascript they have different literal syntaxes. Python chose to use a different literal prefix to its literals ("f" versus "t") and Javascript chose to use a function-call syntax as prefix (`templateString` versus html`templateString` where html is a function in scope).
For the case like here it’s closer to FormattableString that’s what EF Core’s composable FromSql method works on top of. Both address custom interpolation but from different angles / for different scenarios.
> [T-strings] are the pythonic parallel to JavaScript’s tagged templates.
Many languages have similar features.
For instance, Python has the % operator that is a template format that allows interpolating values based on a template string with a variety of printf-like features: https://python-reference.readthedocs.io/en/latest/docs/str/f...
Also, Python has the .format method on strings, which is a template format that allows interpolating values based on a template string: https://www.geeksforgeeks.org/python-string-format-method/
As another example, Python has f-strings that are a template format that allows interpolating values based on a template string: https://www.geeksforgeeks.org/formatted-string-literals-f-st...
Also, you can also find languages like Python that have a rich ecosystem of third party templating solutions. These are often intended for things like rendering entire web pages but many of them have relatively simple ways of using their templating functionality in a fairly reasonable amount of code, if you just want to have a template format that allows interpolating values based on a template string.
So, as you can see, many other languages have this feature, as you can tell from all the examples I have shown you here.
(To spell it out for those who may find this too subtle... somehow... I'm not a fan of this simply because Python has gone from "There should be one-- and preferably only one --obvious way to do it." to "there's half-a-dozen ways to do it and if they are all wrong Python 3.x+1 will introduce a seventh" and I'm just not seeing the benefits worth the tradeoffs here.)
I find this a confusing reply. First, you seem to be confusing eager string interpolation with this lazy/deferred templating feature. I'm specifically asking about languages that have the latter. More confusingly, you bring up the language under discussion as an example... if it already had this feature, they wouldn't be adding a new feature.
Then, you bring up 3rd party templating solutions, but this is off topic, as we are talking about language builtins. I am well aware there are many templating solutions to solve text templating in general.
In summary, I'm specifically asking for examples of languages that allow string interpolation with _deferred/lazy_ processing, which is how I understand this feature. It seems there are a few, but it doesn't seem to be terribly common after reading the other comments.
"I find this a confusing reply."
It's not confusing... it's sarcastic. I don't believe adding literally a third string interpolation method to a language is generally justified, and honestly you can consider it a fourth if you think, as I do, that even "my string " + val + " etc." is itself an interpolation language, especially when combined with things like __str__. I'm afraid you fed me a straight line I couldn't resist; it's not really an answer to you.
What would you say is the alternative?
If I were in charge of Python, I'd probably feature-freeze it. It is already too large, but shrinking it is impractical.
Failing that, because it probably would fail, I would be raising the bar sky-high on new feature additions.
It seems like the project has actually lowered the bar in the last few versions. What was once one of my favorite languages, if not my favorite, is now a language I try to avoid, and approaching one I actively recommend against. When I'm writing it myself it's ok, but using other people's code in it is getting harder and harder, as they keep using all the features.
Hey, at least Python is nowhere near Perl in terms of language creep. I don't need a Period Table of Operators [0] to keep Python straight in my head.
[0] https://www.ozonehouse.com/mark/periodic/
Scala since 2014.
Java 22 had the feature as preview but it was removed in 23, it'll come back after some refinement.
They literally discuss this in the article.
This is a pretty simple and useful feature. I wouldn’t say that it bloats the language too much. Descriptors and metaclasses are much more complicated and have a lot more implications and have been in the language for a veeeeery long time. Is it decades already?
This feature is not complicated, but one must keep every feature that can possibly be seen in code in their head. Even if it is familiar now, what happens when you use the feature in the one small section of code where it fits, nowhere else, and then read that code 2 years later? This is the problem with adding useful features that are only used in a few key places. I'm not saying Go is a perfect language, far from it, but limiting # of features as a general goal is something more languages should strive for IMO.
I am not arguing against that language ideally should have less features. I am arguing with “Python is getting huge”, because it’s huge and has been for many-many years :)
True - cat is well out of the bag at this point
Yeah, Python hasn’t been a simple language for a long time, if ever. That’s probably the biggest misconception about the language - that its friendly syntax implies simple semantics. It’s not true at all.
I would say python in it's entirety is one of, if not the deepest and potentially most complex language I know. C++ is the other contender. The things you could do with metaclasses, multiple inheritance and operator overloading are quite staggering.
I'm just glad you don't have to think or even use this as a normal user of the language, most of the time or at all.
A lot of Python's complexity like descriptors and metaclasses and the so-called cooperative multiple inheritance come from an era where object orientation is gospel and yet people keep finding limitations to what can be accomplished in an OO paradigm and therefore they patch up the language to support ever more complicated use cases.
I have a feeling that if metaclasses and descriptors were to be proposed today it would be laughed out of the PEP process completely.
> I have a feeling that if metaclasses and descriptors were to be proposed today it would be laughed out of the PEP process completely.
I think you need at least one of these. Modern Python eschews metaclasses but descriptors are everywhere. Without them you couldn’t have @classmethod, @staticmethod nor @property - or at least, you couldn’t generalise custom method lookup and all three of those would need to be special cases.
The alternative approach is to drop descriptors and make method lookup much simpler, but then you need metaclasses (see: static methods in Ruby).
If descriptors didn't exist in the language, classmethod, staticmethod, and property would become keywords. Languages like Java and C++ already have static as a keyword; C++ also made its type system have a special pointer to member type; such behavior can be hardcoded without having a flexible descriptor mechanism.
Custom method lookup would just be the same as custom attributes, that is, by the user overriding __getattr__.
I'm truly glad that Go exists for people who like languages that are simple even to the point of frustration and I hope it never changes. But I'm glad that other languages exist for those of us for whom learning some syntax is not a barrier and having convenient ways to do common things is highly valued.
There’s an interesting trade off around maintenance. Python’s stdlib means there’s more consistency across projects and you can assume basic things like error handling which you had to check individually on each Go program, which is the kind of stuff which adds up when you have a lot of small programs.
This is especially noticeable with AWS Lambda where you can have a lot of useful stuff running for years without doing more than bumping the runtime version every year or two, but also that is one highly opinionated architecture so it’s not globally optimal for everyone.
Big discussion (414 points, 10 days ago, 324 comments) https://news.ycombinator.com/item?id=43647716
Maybe it's good to come back to a discussion after a few days, more informed and with a cooler head ?
;-)
It feels a bit like "cheating" that new x-string features are built-in only. It would be cool to be able to do:
A t-string is a literal for a Template object which is a data holder, it doesn't actually do anything, so you would simply call
This is exactly how Nim is. The f-string like equivalent uses a macro called "fmt" which has a short alias "&". So you can say:
The regular expression module does a similar thing with a `re"regular expression"` syntax or std/pegs with peg"parsing expression grammar" and so on. There are probably numerous other examples.In general, with user-defined operators and templates and macros, Nim has all kinds of Lisp-like facilities to extend the language, but with a more static focus which helps for both correctness and performance.
This was the original proposed idea in the PEP (750), but it changed overtime. There is a section in the PEP to explain why it changed to t-strings if you are interested.
PEP 638 has always seemed to me like something of a generalization of the idea. But that really feels to me like a 4.0 feature, or rather something that needs to be designed for from the beginning. (Which is why I've done a bit of that in my own mind...)
True. Then you could use
for each of the given examples and achieve some runtime type safety.And you'd end up with almost no improvement.
If you pass a "t-string" to a framework, it can force escaping.
What you suggest is to rely on escaping by the user (dev), who, if he was aware, would already escape.
Unless you'd suggest that it would still return a template, but tagged with a language.
No, you misunderstood that completely. They would still be like t-strings, but sql-strings are now a different type from html-strings. The escaping would be done by the library that offers the sql"..." functionality.
FWIW the JS equivalent is a template but tagged with a language. It has all the benefits of this template, but IDEs can easily syntax highlight the string. That seems like it would be a bit trickier to do with the Python one which is a shame.
Use a function?
It’s just syntax, like we used to have that later becameIn my own language design (nothing public yet - need to prioritize more practical things at the moment) this is definitely on the menu. Aside from a minimal set, keywords are implemented as compile-time macros; and it's intended that they can be extended, both "natively" and in the implementation language, by writing AST-manipulation code. But the handling of arithmetic expressions, as well as the broader line/block structure, is hard-coded. (I draw inspiration from both Python and the Lisp family.)
meh, the difference between bar"zoop" and bar("zoop") isn't really big enough to be worth it.
I like F strings a lot, but for the most part I think all of the various X-strings should just be classes that take a string as an argument.
What I really don't get is how it's any different than applying whatever function you would apply to the template, on the f-string variables. So instead of:
Why not just: Or even before creating the f-string. Is it just about not forgetting the sanitization/string manipulation part and forcing you to go through that?> Is it just about not forgetting the sanitization/string manipulation part and forcing you to go through that?
This is a very big deal! It's also about centralizing that work. Now that sanitization can occur in the consumer of the t-string (for example, the API to your HTML renderer), rather than in every f-string.
Pretty much, yeah. The article highlights that people were using f-strings directly, and they wanted to provide an alternative for lightweight template/interpolation.
I feel like I'm still missing something when they're saying this about the example(s):
"Neither of these examples is possible with f-strings. By providing a mechanism to intercept and transform interpolated values, template strings enable a wide range of string processing use cases."
As far as I can see, anything you do with the template, you could do before building the f-string or inline as in my intial example.
With f-strings, you cannot write code to determine which portions of the resulting `str` were static and which were dynamic; with t-strings, you can. *
(As to your initial example: it's worth considering what will happen as you compose multiple bits of HTML via nesting to generate a large final page. The developer experience may become... unideal.)
* (barring undesirable hacks with inspect, etc.)
You wouldn’t really do your example, though. If you’re using an f-string, you’d just directly interpolate, because it’s convenient. You wouldn’t use an extra library to properly make it safe, otherwise you’d just use a proper template library and language.
This gives you a convenient middle ground where you don’t need to learn a template library but still get safety. I can’t think of the code right now but I could see this being useful to pass in some dynamic HTML to, say, Django without having to remember to turn off escaping for that section. It can also be convenient for writing raw SQL without having to use prepared strings.
This is pretty cool, if we're porting JS features do we get dictionary unpacking/destructuring next?
I just want this so badly, it's the main reason I drift back to JS:
Another alternative:
The parentheses for this are admittedly a bit tricky.just use this very sane pattern /s:
Zen of Python in 2025:
Python String Formatting in 2025:- t-strings
- f-strings
- %-operator
- +-operator
- str.format()
And $-strings (PEP 292) as well. :-)
I see each of these as distinct but overlapping; I'm (slowly) writing a guide to string formatting with all of these in mind, trying to emphasize when I'd choose one over the other. (fwiw I personally avoid % and + these days; $ is pretty uncommon in practice; f-, t-, and .format() all seem to have good unique uses.)
The last three generally shouldn't be used (`+` is sometimes fine, but not really for formatting), but I doubt we would ever get a py4 that removes them, given the stomachaches that py3 caused. It does feel odd that a t-string is just an f-string without args applied.
It's my understanding that they're still recommended for logging since with f-strings you always pay the formatting cost (but with str.format it's deferred).
t-strings aren't really like these others.
It's definitely true that those four string formatting techniques violate the "one obvious way" advice.
I could scarcely believe this new t-string wasn't a joke. As an occasional, grudging Python programmer, I rue the accretion of string formatters over the years. My code bears the history of successive, imperfectly understood (by me)formatters.
"Situation: There are 14 competing standards...." https://xkcd.com/927/
Why does this need to be a language feature. This could just be a separate library, we could use brackets instead of a letter before a string. I fear, Python is going down the path of C++
It being a language feature gives you controlled access to the lexical scope, such that the template string can refer to variables by name rather than having to pass each value as a parameter. Doing it via parameters is repetitive and/or error-prone.
You CAN get access to the calling scope of a function. Something like:
I did it in uplaybook so that you can do things like: This is an ansible-like tooling with a python instead of YAML syntax, hence the Jinja2 templating.This feature actually can't be a library. It needs to be able to preserve the string before the variables are passed in, which a library would be unable to do because f-strings immediately replace all the values. The whole premise of t-strings is to know which values where "hard coded" and which ones are variables. A library can't possibly know that. And a function call wouldn't have access to the local variables. The only way to do this without language support is to pass `locals()` into every function call.
Now you have me wondering how difficult it would be to have a class that takes a string and parses through globals to find the context it was called from. maybe causing an exception and abusing a traceback? or maybe we just find our own objectid.... gahh I have to try it now, but I'm setting a timer.
It was easier than I thought, but also took longer than I wanted to. turns out the inspect module provides what you need to pull it off.
This dummy example splits a string that it was given, then if one of those values is in the callers context it saves those in self.context, and has an output function to assemble it all together. Obviously this example is not very useful, but it shows how a library could do this as a class or function without the user having to pass in locals().
This looks like a performance nightmare, and would likely never have IDE integration. So I guess I was wrong. It could be a library. But will be much better supported officially.
You don't have to go through all the globals, you just have to get the caller's namespace, which is fairly simple. See e.g. https://stackoverflow.com/a/6618825/27426
For this reason, I think it's not true that this absolutely had to be a language feature rather than a library. A template class written in pure Python could have done the same lookup in its __init__.
That is exactly what I ended up doing:
https://news.ycombinator.com/item?id=43752889
It's not as simple as "more features" == "closer to C++". Features are not equal to each other in terms of impact on language complexity.
t-strings don't interact with anything else in the language; they, as you yourself pointed out, could almost be an isolated library. That makes them low impact.
This is also true syntactically; they're just another type of string, denoted by "t" instead of "f". That's easy to fit into one's existing model of the language.
Moreover, even semantically, from the point of view of most language users, they are equivalent to f-strings in every way, so there's nothing to learn, really. It's only the library writers who need to learn about them.
Then we have to consider the upsides - the potential to eliminate SQL and HTML injection attacks. The value/cost is so high the feature a no-brainer.
If it's not a language feature, there's always a risk of fragmentation. Some people won't use it because it adds another dependency, that means fewer programmers will be familiar with it. Others will come up with their own, slightly incompatible implementation. See for example Perl and its various Object Orientation frameworks (Moose, Mouse, Moo, Mo, M, Mojolicious comes with its own...)
Other languages have a policy of prototyping such things out of core, and only adding it to the core language if it gains traction. Of course that works better if the language has a mechanism for extending the syntax out of core.
putting it in the standard library would handle those issues.
Counterpoint: It's good to add well designed and useful features to programming languages.
Because JS has it in the form of tagged template literals.
We have a handful of ways to create strings now.
Meanwhile, pytest is still not part of the standard library.
Maybe this could be useful to libraries like psycopg3 to use something more simple/natural instead of this:
https://www.psycopg.org/psycopg3/docs/api/sql.html
(while I also agree it gets crowded with yet another string prefix)
That's the exact use case. Basically these are syntactic sugar for a very similar function signature.
Hi! I wrote this. :-)
I'm a little late to the conversation (and a bit surprised to see this trending on HN) but am happy to answer any questions; I'll try to pop in throughout the day.
Hi. I come from a JavaScript background.
I am wondering what is the reason behind not using a similar syntax to JavaScript? Seems simpler to me.
The PEP originally started with a similar-to-javascript syntax but over time we decided it wasn't the right way to expose these ideas in Python. There's more detail about why this approach was rejected in the PEP: https://peps.python.org/pep-0750/#arbitrary-string-literal-p...
Would be interested in inclusion of PEP 292 [1] in your discussion here, which introduced `string.Template`. Is this Template going to be deprecated?
[1] https://peps.python.org/pep-0292/
PEP 292's `string.Template` will remain; there are no plans to deprecate it.
PEP 750's `string.templatelib.Template` is a separate and unrelated type. Amongst many differences, unlike PEP 292, `Template` has a literal form too.
I'm hopeful that the confusion will be minimal; in practice, PEP 292 (aka $-strings) is used only in specialized cases, like flufl.i18n, a really deep I18N framework.
This is super cool, thank you.
Landing in 3.14? Nice, but also oof, that's probably not getting to my employer's codebase for a year or two. And it sounds like it could really solve some problems for us, too.
Paging asottile - any plans to make a `future-tstrings`? :)
`future-fstrings` (https://pypi.org/project/future-fstrings/) was a real QOL improvement for our team for a year or 2 around 2019 before we got onto Python 3.x.
Some prior art: https://pypi.org/project/tagged/
In fact, the repo of a companion project from the author has the ticket that spawned the work leading to t-strings: https://github.com/jviide/htm.py/issues/11
I asked asottile for his opinion on PEP 750 (or rather, a previous version of it), and he wasn’t really in favor. So I think it’s unlikely that we’ll see a future-tstrings from him. :)
How do these interact with i18n? Can I load a translated t-string with `_(t"")` from a .po file? Can it include variable names and arbitrary code inside lambdas?
I suggest you read the PEP, there is a section on i18n. Short version is it's not designed for this use case.
As for variables and arbitrary code/lambdas, yes: t-strings can do that, just like f-strings
> t-strings evaluate to a new type, `string.templatelib.Template`
> To support processing, `Template`s give developers access to the string and its interpolated values before* they are combined into a final string.*
Are there any use-cases where processing a Template involves something other than (i) process each value, then (ii) recombine the results and the string parts, in their original order, to produce a new string? In other words, is the `process_template` function ever going to be substantially different from this (based on `pig_latin` from the article)?
I haven't seen any examples where the function would be different. But if there aren't any, it's strange that the design requires every Template processing function to include this boilerplate, instead of making, say, a `Template.process` method that accepts a `process_value` function.Templates don't even have to be processed into a string. The article shows an example where the Template is processed into an HTML mini-DOM. It's maybe not obvious because the DOM object is immediately stringified to show sample output, but you could imagine manipulating the DOM object in a few more steps before stringifying it, or maybe you are running in WASM in a browser and using that mini-DOM directly as a Virtual DOM passed to JS to work with.
Also, in addition to the other SQL example using "?" to fill in the "holes" for parameters in an SQL friendly way, some DBs also support named parameters, so the "hole" in the string form might be naively replaced with something like `f"@{item.expression}"` and that also forms the key in a dict to pass as parameters. (You'd want to make sure that the expression inside the template is useful as a parameter name, and not something more exotic like {1 + 3} or {thing for thing in some_list}, in which cases you are probably auto-assigning some other parameter name.)
Nearly everything you just described is being worked on. It's amazing how accurately you have described it. We hope to demo and explain at PyCon US.
There are a lot of examples about SQL in comments. In the SQL case you want something like:
(of course it would be more nuanced, but I hope you get the point)Yes that makes sense, thanks.
Also, my comment was about the amount of boilerplate required, but that can be vastly reduced by writing `process_template` in a more functional style instead of the highly-imperative (Golang-like?) style used in the article. The first `process_template` example is just:
And the second is something like:>But if there aren't any, it's strange that the design requires every Template processing function to include this boilerplate
Other replies gave examples of other use cases. But the neat thing about Python is that you don't need to "include this boilerplate" for the common cases. It can be wrapped up in a decorator (which could be included in `templatelib`). Or, as you say, in a method on the Template class.
I think I'd implement it as a generator, calling `process_value` (defaulting to `str`) on the Interpolations, so that the caller can still do more with the results (or just `''.join` them).
But these are separate considerations; nothing prevents implementing them later, or indeed adding them to the implementation before the 3.14 release.
I wish they added the same thing JS has, where this "string literal prefix thingy" can be user-defined.
html`<p>${value}</p>` will actually run the function html(template). This means you can use this to "mark" a function in a way that can be detected by static analysis. Many editors will, for example, syntax highlight and lint any HTML marked this way, same with SQL, GraphQL and probably some others too.
The PEP was initially proposed this way. But due to various reasons, making it an open namespace was considered to overcomplicate the language (read: understanding coffee when reading it). Alternatively, there doesn't really seem to be much loss of ability with t-strings. Libraries can require a template as it's accepted type, instead of having to invent their own custom type and named template.
For the record the JS thing desugars to the exact same as the Python thing, so it is no more or less safe to do the syntax highlighting in Python as it is in JS.
It desugars similarly, but the Python version doesn't have a name. Any t-string is a t-string, there's no HTML t-string or SQL t-string or anything like that. It's just a t-string you can pass to a function:
In JS, the string has a prefix that can differ between languages, e.g.: and so on. See the difference?Except your labels are incorrect because neither `html_string` nor `sql_string` are strings, they're both Template objects, and the sink function is the one which processes it. No processing has happened to them by the end of the snippet, beyond creating the template object itself.
Sure, choose different variable names, who cares. The essential difference is that the language is referenced at the declaration site, not the usage site, which makes the syntax highlighting far easier.
Please engage with my point instead of criticizing trivialities.
> Please engage with my point instead of criticizing trivialities.
Your complete misunderstanding of what's happening is not a triviality.
> The essential difference is that the language is referenced at the declaration site, not the usage site, which makes the syntax highlighting far easier.
Javascript has no built-in template tags beyond `String.raw`. If tooling has the capabilities to infer embedded language from arbitrary third party libraries, I would hope they have the ability to do utterly trivial flow analysis and realise that
means the template string is pretty likely to be HTML content.Yes. It's a weak heuristic, but it's EXACTLY the same weak heuristic that JS is applying!
IN other words, since custom template tags in JS *are literally just function calls* when a JS environment syntax highlights the code as HTML it's doing so based on an extremely weak heuristic (the identifier for the interpolation function is named "html"). Both Python and JS have the same problem.
I think his point would be clearer if we focused on typing the usages statically. Consider `html(this_is_a_sql_template)` vs. `html"SELECT * FROM ..."` or `thing_that_consumes_html_template_type(oops_a_sql_template_type)`.
Come on, you're just being rude for no good reason. A badly chosen variable name doesn't show a "complete misunderstanding". Yes, the variable should have been named `html_template` instead of `html_string` - how often do I have to acknowledge this before you accept it?
And it's obviously more complex to do syntax highlighting when the declaration site and usage site are possibly split apart by variable assignments etc. Yes, in the case you showed syntax highlighting is easy, but what if the `html` function takes more parameters, doesn't take the template as the first parameter, etc? There's a lot of possible complexity that tagged template literals don't have. Thus they are easier to do highlighting for. This is objectively true.
Tagged template literals in JS have all that complexity. All of it. That tools you trust lie and pretend that's not the case doesn't make the language spec say anything different
No, they literally don't, because they don't support these features!
You can't split apart the declaration of the template literal and the "tagging". The tag is always part of the declaration, which it doesn't have to be in Python, as I've showed.
You can't pass additional parameters to the tag function, it's always just the template & values. In Python, you can pass as many parameters as you want to the usage site, e.g.
It just sounds like you don't know JS very well, because in JS you can definitely split apart the declaration and the tag with ease. The thing implementing the tag is just a function after all: https://jsfiddle.net/rd2f1kot/
Sorry, but do just not want to understand what I'm talking about? Your example doesn't show what you're saying it does.
In Python you can do this:
This is simply not possible in JS, because the template literal always must have the tag attached. You can't split them apart. If you try: you already have a string in the `template` variable. There's no access to the individual values anymore. It's already done. It's a string. No template. You can't pull apart the literal declaration and the tagging.Are we now done with this ridiculous game of deliberate misunderstandings?
Sorry for some reason I can't reply to the deeper posts. Depth limit I guess. You asked which editors could support syntax highlighting once the tag and the content that needs to be syntax highlighted are split apart. Currently there are none, though I am writing one that will be able to.
Python should be able to detect the magic syntactic pattern just the same way JS does and use that for syntax higlighting. In JS the magic syntactic pattern for triggering HTML syntax highlighting is:
In Python the magic syntactic pattern would be: My point in showing that JS counterexample was to demonstrate that the real reason people don't do that kind of thing isn't that they can't, it's that they like having syntax highlighting. That means the approach should work just as well in Python. This is the case even though the heuristic involved is very weak. Changing the name of the identifier or breaking into multiple expressions would be enough to break the heuristic in either language, which is why I think it's a really weak heuristic and dangerous pitfall for developers who might mistake the coloring for information about how the runtime sees that data (which is normally what syntax highlighting is for)If I may give you some advice: you could have made this point without telling me I don't know JS well, without misrepresenting me and without wasting both of our time by just making this point from the start. Because you are technically correct that, in the limited case of a template immediately being passed to a function, syntax highlighting can use the same heuristics that are currently used in JS. But, as I assumed previously, in doing so you're simply ignoring my point: syntax highlighting for tagged template literals isn't applicable to only a subset of uses because they are by design less complex. Your counter example was using a different template tag and not using the actual tag as a template tag, which obviously doesn't work in the context of syntax highlighting for that template tag.
Had you not played games with this technicality, we could both have saved a lot of time. Hope you had fun I guess.
I just don't see the same thing as you. This is what you say is simply not possible:
So now we have moved from "tagged template literals have all the complexity of Python templates" to "if you define a specific template tag and use that, you can replicate one feature of Python templates". No mention of the other example I mentioned (which, by the way, wasn't an exhaustive list).
Now, I'll just ignore that you're still deliberately misrepresenting me in the hopes of gaining some actual knowledge from this discussion - please show me: which editor supports syntax highlighting for the specific example you just mentioned? After all, that was the topic of discussion - not whether it's possible to delay "tagging", but whether it's easier to do syntax highlighting for JS template tags. Please show me an existing editor, IDE or git repo for a tool that would highlight your specific example based on the `html(...template)` call in line 4 (so no heuristics based on line 3). Surely you're not just throwing random examples at the wall to see what sticks, right? You've actually been following the discussion and are arguing in the context of it?
Oh now I can reply. Thread continued above: https://news.ycombinator.com/item?id=43754585
JS references the language at the usage site, exactly like Python. There is no difference here in how the two languages behave.
No, it doesn't, it references the language at the declaration site, because the declaration site always is the usage site. You can't split them apart. You can split them apart in Python - see the example in my first comment.
The JS version actually desugars to something much more primitive, and less convenient: in JS a template tag receives one parameter which is an array of n+1 strings, and then n parameters for the interpolated values, and has to iterate alternatively on both sequences.
You can do that in python by accessing the `strings` and `values`, but I expect most cases will simply iterate the template, yielding a unified typed view of literal strings and interpolated values.
> If you’ve worked with JavaScript, t-strings may feel familiar. They are the pythonic parallel to JavaScript’s tagged templates.
The syntax is template literals, not just "tagged templates". Which is a huge difference: template literals still act as real strings. They don't need a tag prefix to work, you have the option to tag them if and when needed.
As far as I understand it, t-strings can't do that. They're not strings, and you can't even coerce them into strings, you have to run them through a processor before they become a string. So they're nothing like JS's template literals, they're syntactic sugar for forming "an instance of an object that needs to be passed into a function that returns a string".
So I don't look forward to folks preferring f-strings over t-strings even when they really shouldn't, simply because "having to constantly convert them from not-a-string to a string is a hassle". If only they'd worked like JS template literals.. that would have been fantastic.
Looks useful for embedding interpreters
Debate around the usefulness aside, are there any linter rules for warning about f-strings in light of this? I can easily see where mistaking one for the other would cause problems. For context, I'm thinking specifically about tools like Black and MyPy.
This will be trivial to implement (it is just a check against the type that t-strings return), so I'm sure it will be done, if it hasn't already.
The runtime and static type of the template string is different to fstring (Template vs str). Unless you’re passing them somewhere that accepts strings, you can’t go wrong.
I actually quite like the simplicity of this design over tagged literals in JS.
Will it be a performance boost if Django's template engine started using t-strings internally?
Why would mere syntax make a difference?
No, I mean that if Django's template engine internally switched over its implementation to enable templating (as a functionality) to use these new t-strings. If t-strings are more highly optimized within Python, it could mean a performance boost.
Nice, so it's a kind of "limited DSL" inside python that's easy to extend
The PEP originally portrayed this as "for DSLs"
https://peps.python.org/pep-0750/#approaches-to-lazy-evaluat...
name = "World"
template = t"Hello {(lambda: name)}"
This looks cool
TL;DR: like f-strings, all {foo} expressions in the t-string are evaluated immediately, but instead of immediately concatenating everything into a single result string, the t-string evaluation returns a Template object that keeps the interpolation results and the surrounding strings separate. This lets subsequent logic decide whether the interpolation results need any special escaping before concatenating them with the strings around them.
In other words, t-strings are basically f-strings where the final concatenation is delayed. And indeed, you can trivially implement f-strings using t-strings by performing a simple, non-escaped concatenation step: https://peps.python.org/pep-0750/#example-implementing-f-str...
> This lets subsequent logic decide whether the interpolation results need any special escaping before concatenating them with the strings around them
This sounds like unnecessary fluff in what was supposed to be a simple language. I'm worried Python is turning into C++42 with 65535 ways to do one simple thing.
Why not just:
Nice and simple.It's easy to forget the `esc` function. How does the recipient check (or type check) that it was called in all the right places?
Most DBs support parameterized queries which can be cached for performance. How do you pick out the parameters from that and replace those parts of the strings with the DB's parameter placeholders?
Looks very similar, but execution engine has access to all the individual parts, making it very easy to add placeholders such as: Or even (if the DB supports it), has access to the expressions inside the string and can use named parameters: That's really nice for debugging, depending on your DB engine.In every DB engine that supports it, parameterized SQL is even safer than escape syntaxes because parameters are passed in entirely different parts of the binary protocols and don't need to rely on just string manipulation to add escape sequences.
While your code is a valid alternative way to implement @haberman's description, the feature is actually much more flexible.
The "subsequent logic" has full access to the interpolation results and strings. Not only can it escape the results, it can do whatever it wants to them. It can also do whatever it wants to the strings, and then combine everything in any way it likes - it's not even necessary that the final result is a string.
The other PEP example shows generating HTML attributes from a passed-in dictionary. HTML has a number of places where this is helpful, if you have original data.
by making it a generic `t` you lose explicit syntax highlighting. Where something like JS template`string` could determine which syntax to use based on the template value.
I supposed when assigning it to a, variable: SyntaxRecognizableTemplate, you could give it the hint necessary.
was this discussed in the PEP?
*edit: reading the PEP-750[1] it doesn't seem like it..
[1] https://peps.python.org/pep-0750/#the-interpolation-type
We pushed these questions out of the PEP to keep its scope constrained. (See https://peps.python.org/pep-0750/#mechanism-to-describe-the-...)
But yes, the PEP leaves open an important question: how will tools decide to work with common types of content in t-strings, like HTML or SQL?
There are simple approaches that can be taken in the short term (content sniffing) and more robust approaches (type annotations, perhaps) that will take time and the broader tooling community to develop.
It was discussed in the first revision and discussion of the PEP. The decision was made to move that to follow-on work, as we discovered more about what tooling needs.
As an example, I was excited about using `Annotated` on the function to indicate the language it expected. Turns out, a lot of linters know nothing about the type system.
I had something like this implemented for safely executing shell commands, by using metaprogramming:
name="A$ron"
z("echo Hello {name}")
Note that this is not an f-string. The z function expands the variables by parsing this string and accessing its caller's local variables.
https://github.com/NightMachinery/brish
Not sure about introducing yet another string prefix. Between f-strings, raw strings, and i18n stuff, it’s already getting crowded. Curious how readable this will be in large codebases.
I'm of the opposite opinion. Let's set the prefixes free!
This was considered and rejected: https://peps.python.org/pep-0750/#arbitrary-string-literal-p...
How would this be different from a function sql() that operates on one of these new t-strings?
The syntactic sugar of changing it from sql(t"...") doesn't seem particularly valuable. The novel thing about t-strings is that they change the parsing at compile-time.
> The syntactic sugar of changing it from sql(t"...") doesn't seem particularly valuable.
It's valuable because:
- IDEs could then syntax-highlight SQL inside of SQL strings and HTML inside of HTML strings
- You can't accidentally pass an HTML string to your SQL library
It’s different from a function the same way f”” is different from f(“”) and t”” is different from t(“”)
There’s nothing stopping you from building a Python function that parses a string looking for {} and then searching globals for those variables. And you can extend that to also do some code execution and formatting.
To me the real sugar of f-strings is that the editor knows that it’s a template and not just a string. Expanding this to having SQL and regex syntax highlighting, linting and code formatting inside my Python code is a pretty cool prospect.
It wouldn't be different, but it would be more convenient because we no longer have to count the number of %s, you'd put the variable inline.
That's... already the case of t-strings?
Yes, that's my point. The GP was already talking about a t-string.
dmurray was comparing a hypothetical sql"..." with sql(t"..."). There are no %s either way.
This is how JavaScript does it with tagged template literals.
Your sql there would just be a function that receives the array of strings/values and returns whatever.
This is what JS does with tagged template literals. https://github.com/dsego/sql_tag
Are there string prefixes for i18n stuff?
They're probably talking about the convention of using _ as an alias for `translate`
"Yet another" is not my main worry
The concept of prefixes itself feels a little deviant from readable code that is close to human language -- which is the spirit of Python
Additionally, it will probably be confusing that it is called a t-string but it is actually a constructor for a Template object and not string at all. I would rather see a new special term `template` than this.
The single letter f or t does make it unnatural to read, but if it were sql"..." or html"...", I think that would help with that.
Should have been a keyword.
As should raw and format.Would like to see Django fully use these to replace a lot of it's own complicated template syntax.
As an example of t-strings helping SQL, Phil Jones published a package [1] that illustrates how it could work.
[1] https://github.com/pgjones/sql-tstring
So should f strings now go away? They are just a special case of t strings.
Also, don’t get me started on g strings.
> So should f strings now go away?
No
> They are just a special case of t strings.
Not really, because they produce a string right away instead of a template.
No. F-strings should still be the default. T-strings aren't usable directly, and don't have a `str` like API, so they aren't even compatible, intentionally.
Honestly think this is a more useful feature and elegant solution than the walrus operator that was added. Formatting query strings has always felt messy especially with different DBs having their own non-standard ways of doing it.
How does this Python docs example work with t-strings?
Can SQLite3 cache the query as it does now?It seems to me that it would be straightforward to parse a Template's parameters back into ? placeholders?
No, it would fail upstream before the template post-processing even begin.
The template object itself could not be formed because the each name must be a visible variable name.
This is the same error that you would get with an f-string that contained an undefined variable name.
I'm intrigued but confused by your comment. I think it could be done like this:
Then the Template object would have a parameter that is a list, and it could turn that into the right number of "?"s to pass into the database driver along with the data.What am I missing?
Lowercase f is very visually similar to t. Hopefully no bugs occur from a typo
Seems pretty neat so far, but I don't understand the motivation behind not allowing you to call str(template) and get the template as a normal string. I could imagine it being very useful to be able to gather up the template itself in a string to do stringy things with.
The only reason I could imagine, is if you are trying to protect developers from themselves, which kinda goes against the "we're all adults here" mentality that makes Python so great. I suppose it's easy enough to add that functionality, but come on.
The traditional way at the megacorp for something like that is unsafe_str_DO_NOT_USE_OR_YOU_WILL_BE_FIRED(template)
Call me a monarchist, but I think Python has changed for the worse ever since Guido van Rossum stepped down.
I don't think things would be very different if Guido were still BDFL. He's second-author of the t-strings PEP, and has been an author of most other major PEPs in recent releases (including the walrus operator, PEG parser, and pattern matching).
In what ways has it changed for the worse?
It's so fascinating to see so many people, when faced with a simple and useful programming language feature, get all up in arms about how it's the end of the world for that language.
I honestly feel like a lot of people just seem bored and looking for stuff to be mad about.
I for one think python has never been better.
is this just a copy paste of the PEP announcement?
what's the tldr difference between this and .format ?
It’s injection safe and compostable, and the resulting object retains the original values you interpolate in. This makes it useful for building SQL queries with prepared arguments for example.
Compostable? The Green New Deal has gone too far.
It’s on a meta level: instead of formatting the string, it returns an object that contains both the format string and its argument. Library author can then implement whatever format function they want, for example one that escapes the interpolated strings.
f-strings are syntactic sugar for .format, e.g.:
are equivalent.t-strings are actually not strings, but Template objects, giving access to both the templating string and the parameters for processing. Sibling comments describe it as a custom .format implementation in that sense - it's f-string-like sugar where you're also allowed to take control of the .format function that it's sugar for.
f-strings are actually translated into formatting and concatenation of the pieces at a bytecode level; they don't call `.format` under the hood and thus aren't what I'd call "syntactic sugar" in the traditional sense. But yes, the equivalence holds.
t-strings, of course, (will) translate at the bytecode level into instantiation of the Template object (and whatever code is needed to compute the Interpolation values in-place).
It’s custom .format implementation. (You access the placeholder and value and produce the string)
If this is just for sql queries ... it'd be overkill especially where you need to compare the usual PREPARE statements with the hassle of keeping everyone on 3.14 and above.
It's also for logging:
builds a new string every time the interpreter hits this line. Whereas passes a Template to the debug() function that bails out if debug mode is not on and doesn't build a string.Could also be used to prevent html injection.
It's for SQL, HTML, and shell. But idk how solving injection, a top on the OWASP list forever is considered "overkill".
Sure, this avoids issues with SQL injections. However, I have a hard time imagining any developer who would both make such fundamental errors with f-strings currently and also switching to this option when it ships.
Seems like a self selection which renders this meaningless, to some extent :/
> However, I have a hard time imagining any developer who would both make such fundamental errors with f-strings currently and also switching to this option when it ships.
t-strings are a different type (both static and dynamic), f-strings are not. So t-strings can be mandated at the API level, forcing the developer into "proper" usage.
That is, you need third-party tools to differentiate between
and That is not the case when it comes to t-strings, `some_call` can typecheck internally that it got a t-string, and reject regular strings entirely.Although some space probably needs to be made for composing t-strings together in case of e.g. runtime variation in items or their count. Facetting for instance. I don't know if that's a native affordance of t-strings.
But that would require any SQL library you're currently using to make the breaking change of no longer allowing strings.
sqlalchemy doesn’t really accepts strings - if you do, you need to pass them into a “conn.execute(text(…))”, so end users should not face a breaking change.
Yep. Probably worth it. You could also do this with a monkeypatch method to "opt in" to this change.
Yes. That's exactly what will happen, over time.
Yes?
Eventually, you won't be able to pass a string into the commonly-used DB libraries.
I feel like this can be solved another way. S=f”my good code #### {potentially_evil_user_input} #### my good code again” then work around the ####. Of course, even better, S=evil_user_input and do a scrub on S first.
I really was on the side of being generally willing to accept new python features, but this is getting ridiculous. What an utterly pointless thing to bloat the language with. At this point my moving to clojure as my first line language of choice is only accelerating.
This is of the category "things I wouldn't want to use even for the specific hyper niche things they're intended for". What even does a "t-string" represent? Because it's clearly not a string of any kind, it's a weird kind of function call notation. The programmer sees something that looks like string formatting, but the program executes some arbitrary procedure that might not return a string whatsoever.
For me, this is the best feature to land in python for 6 years. JS has had this and it allows my code to be completely safe from SQL injection, which is an absolutely incredible feature, given SQL injection has been the #1 vulnerability for a long time.
Thanks Nick for this response and all the time you've spent explaining. It's funny, I looked back at the comments on f-strings before they landed. They also got similar complaints about bloat. And yet, my uneducated guess: very popular.
But this doesn't prevent SQL injection, does it? It adds a grammar feature that you can then use to build SQL injection prevention following a novel idiom. Someone still needs to write the actual SQL building logic somewhere!
I don't think this is the right idiom for doing this. Frankly I don't think SQL query generation should look like string templating at all!
The sell seems to be "now you can write code that looks like an SQL injection vulnerability, without it actually being vulnerable!". I'd rather write code that isn't a vulnerability, and doesn't look like one, and doesn't have to bend the language grammar either.
I'd rephrase the sell as "now people who are used to doing the wrong thing and risking vulnerabilities can do the right thing without any extra effort," with a footnote about the difference in types allowing libraries to force the change.
What? Maybe in 2005. This is just strictly worse than parameterized because now you are also wasting time "escaping" strings which perpetuates the whole mixing data & query thing nobody even wants anymore.
It's like even the one case identified nobody has even thought all the way through. Now your SQL library only accepts t-strings, I get an obscure error passing in a simple static query. Ah yes, put the useless t on it. That sorted, now the SQL library escapes all the parameters it wasn't previously doing, to then hand the final unique query to the actual underlying SQL library which would much rather have the parameterized one so it can cache parsing. Jesus.
The "bloat" is that you can now put the letter "t" in front of a string, rather than "f" or "r"?
I'm not the OP but I'm guessing with OP moving to Clojure that the bloat is basically any special syntax or special language feature that basically boils down to a function call or a let statement. A lot of functional programming languages have minimalistic expression syntax and they are just as expressive as OP needs them to be.
It's a pattern moreso than this specific feature. There's an important qualitative distinction for me between something that's on the level of a library (even if it's the stdlib) on the one hand and first class features (ie things that require parser level support) on the other.
Python has historically been very conservative about this but in recent years has had one controversial language extension after another, while parts of the language that actually need love are left to languish IMO.
I wanna be very clear that this is me changing my mind -- I was (still am) very on board with the highly controversial assignment expressions ("walrus operator") for instance.
I don't have much faith about what the Python language will look like if you project the current rate of changes forward 10, 15, 20 years. It really doesn't help that I consider this new thing an active antifeature.
I feel like this is begging the question... This t-string feature is only controversial inasmuch as comments like yours here are criticizing it as being controversial...
I enjoy f-strings, I guess some people need these.
And I love Python but, having been through 2->3 ( occasionally still going through it! ) whenever I see a new language feature my first thought is "Thank goodness it doesn't break everything that went before it".
Yeah but it's been 17 years, maybe it's time to put the PTSD behind us. We're almost at a point where the current generation of programmers wasn't even programming when that happened.
> We're almost at a point where the current generation of programmers wasn't even programming when that happened
I've been programming with Python since 2006, I think most of the systems were based on 2.4 at the time. I've been one of those who switched to Python 3 somewhat late, waiting for some major libraries to ship python 3 packages - celery and Twisted were one of the biggest holdouts - so I remember that the first project where all my dependencies were ready for python 3 was around 2015.
This is to say: even seasoned developers who were conservative around the migration have spent more time working with Python 3 than Python 2. There simply is no reason anymore to be talking about python 2.
The last time I touched a large Py2 project was in 2018 when I ported it to Py3. So, I have 18 years of Py2, probably 6 years of overlap, and 7 years of pure Py3. That means I still have a lot more Py2 than Py3 time.
Buuuttt, I'm so over the transition. It’s ancient now and I agree that we can stop fretting about it.
Python2 code didn't disappear when Python3 came out. At my work we're _still_ occasionally having to help people migrate code that was written for python2
Also my experience, alas.
We are not completely Post Traumatic Python2 Stress yet, I am afraid.
Bad decisions can have looong-term repercussions.
We're at a point where the current generation of programmers weren't even _alive_ when that happened.
Yes, Python 3.0 was released 17 years ago. But the transition from Python 2.x was only completed with 2.7’s end-of-life, 5 years ago.
"It's still supported" is a strange metric for this. I mean, ActiveState still provides Python 2.7 builds with (paid) support.
And Ubuntu ESM got used as an excuse/"life support" for python 2 via 16.04 until horrifyingly recently. (With a layer of "you can still get ESM for 14.04, we're not that far behind" :-)