Write a template string "tutorial" in the "fancy input output formatting"

Author: davepeck

Doc/library/string.rst

@@ -799,7 +799,7 @@ Template strings ($-strings)
 
    The feature described here was introduced in Python 2.4.  It is unrelated
    to, and should not be confused with, the newer
-   :ref:`Template Strings <template-strings>` feature and
+   :ref:`template strings <template-strings>` feature and
    :ref:`t-string literal syntax <t-strings>` introduced in Python 3.14.
    T-string literals evaluate to instances of a different
    :class:`~string.templatelib.Template` class, found in the

Doc/library/string.templatelib.rst

@@ -11,8 +11,9 @@
 
 .. seealso::
 
+   :ref:`T-strings tutorial <tut-t-strings>`
    :ref:`Format strings <f-strings>`
-
+   :ref:`T-string literal syntax <t-strings>`
 
 
 .. _template-strings:
@@ -22,47 +23,12 @@ Template strings
 
 .. versionadded:: 3.14
 
-Template strings are a generalization of :ref:`f-strings <f-strings>`
-that allow for powerful string processing. The :class:`Template` class
-provides direct access to the static and interpolated (substituted) parts of
-a string.
-
-The most common way to create a :class:`!Template` instance is to use the
-:ref:`t-string literal syntax <t-strings>`. This syntax is identical to that of
-:ref:`f-strings`, except that the string is prefixed with a ``t`` instead of
-an ``f``. For example, the following code creates a :class:`!Template`:
-
-   >>> name = "World"
-   >>> greeting = t"Hello {name}!"
-   >>> type(greeting)
-   <class 'string.templatelib.Template'>
-   >>> list(greeting)
-   ['Hello ', Interpolation('World', 'name', None, ''), '!']
-
-The :class:`Interpolation` class represents an expression inside a template
-string. It contains the evaluated value of the interpolation (``'World'`` in
-this example), the original expression text (``'name'``), and optional
-conversion and format specification attributes.
-
-Templates can be processed in a variety of ways. For instance, here's a
-simple example that converts static strings to lowercase and interpolated
-values to uppercase:
+Template strings are an extension of :ref:`f-strings <f-strings>`
+that allow for greater control of formatting behavior. The :class:`Template`
+class gives you access to the static and interpolated (in curly braces)
+parts of a string *before* they are combined into a final string.
 
-   >>> from string.templatelib import Template
-   >>> def lower_upper(template: Template) -> str:
-   ...     return ''.join(
-   ...         part.lower() if isinstance(part, str) else part.value.upper()
-   ...         for part in template
-   ...     )
-   ...
-   >>> name = "World"
-   >>> greeting = t"Hello {name}!"
-   >>> lower_upper(greeting)
-   'hello WORLD!'
-
-More interesting use cases include sanitizing user input (e.g., to prevent SQL
-injection or cross-site scripting attacks) or processing domain specific
-languages.
+See the :ref:`t-strings tutorial <tut-t-strings>` for an introduction.
 
 
 .. _templatelib-template:
@@ -82,28 +48,45 @@ reassigned.
    :param args: A mix of strings and :class:`Interpolation` instances in any order.
    :type args: str | Interpolation
 
-   While :ref:`t-string literal syntax <t-strings>` is the most common way to
-   create :class:`!Template` instances, it is also possible to create them
-   directly using the constructor:
+   The most common way to create a :class:`!Template` instance is to use the
+   :ref:`t-string literal syntax <t-strings>`. This syntax is identical to that of
+   :ref:`f-strings` except that it uses a ``t`` instead of an ``f``:
+
+   >>> name = "World"
+   >>> template = t"Hello {name}!"
+   >>> type(template)
+   <class 'string.templatelib.Template'>
+   >>> template.strings
+   ('Hello ', '!')
+   >>> template.values
+   ('World',)
+
+   While literal syntax is the most common way to create :class:`!Template`
+   instances, it is also possible to create them directly using the constructor:
 
    >>> from string.templatelib import Interpolation, Template
    >>> name = "World"
-   >>> greeting = Template("Hello, ", Interpolation(name, "name"), "!")
-   >>> list(greeting)
+   >>> template = Template("Hello, ", Interpolation(name, "name"), "!")
+   >>> list(template)
    ['Hello, ', Interpolation('World', 'name', None, ''), '!']
 
-   If two or more consecutive strings are passed, they will be concatenated into a single value in the :attr:`~Template.strings` attribute. For example, the following code creates a :class:`Template` with a single final string:
+   If two or more consecutive strings are passed, they will be concatenated
+   into a single value in the :attr:`~Template.strings` attribute. For example,
+   the following code creates a :class:`Template` with a single final string:
 
    >>> from string.templatelib import Template
-   >>> greeting = Template("Hello ", "World", "!")
-   >>> greeting.strings
+   >>> template = Template("Hello ", "World", "!")
+   >>> template.strings
    ('Hello World!',)
 
-   If two or more consecutive interpolations are passed, they will be treated as separate interpolations and an empty string will be inserted between them. For example, the following code creates a template with a single value in the :attr:`~Template.strings` attribute:
+   If two or more consecutive interpolations are passed, they will be treated
+   as separate interpolations and an empty string will be inserted between them.
+   For example, the following code creates a template with a single value in
+   the :attr:`~Template.strings` attribute:
 
    >>> from string.templatelib import Interpolation, Template
-   >>> greeting = Template(Interpolation("World", "name"), Interpolation("!", "punctuation"))
-   >>> greeting.strings
+   >>> template = Template(Interpolation("World", "name"), Interpolation("!", "punctuation"))
+   >>> template.strings
    ('', '', '')
 
    .. attribute:: strings
@@ -156,7 +139,8 @@ reassigned.
        ('World',)
 
        The ``values`` tuple is always the same length as the
-       ``interpolations`` tuple.
+       ``interpolations`` tuple. It is equivalent to
+       ``tuple(i.value for i in template.interpolations)``.
 
    .. method:: __iter__()
 
@@ -175,7 +159,7 @@ reassigned.
 
    .. method:: __add__(other)
 
-       Concatenate this template with another template, returning a new
+       Concatenate this template with another, returning a new
        :class:`!Template` instance:
 
        >>> name = "World"
@@ -190,11 +174,13 @@ reassigned.
        an :class:`!Interpolation` (to treat it as dynamic):
 
        >>> from string.templatelib import Template, Interpolation
-       >>> greeting = t"Hello "
-       >>> greeting += Template("there ")  # Treat as static
+       >>> template = t"Hello "
+       >>> # Treat "there " as a static string
+       >>> template += Template("there ")
+       >>> # Treat name as an interpolation
        >>> name = "World"
-       >>> greeting += Template(Interpolation(name, "name"))  # Treat as dynamic
-       >>> list(greeting)
+       >>> template += Template(Interpolation(name, "name"))
+       >>> list(template)
        ['Hello there ', Interpolation('World', 'name', None, '')]
 
 

Doc/tutorial/inputoutput.rst

@@ -34,13 +34,28 @@ printing space-separated values. There are several ways to format output.
      >>> f'Results of the {year} {event}'
      'Results of the 2016 Referendum'
 
+* When greater control is needed, :ref:`template string literals <tut-t-strings>`
+  can be useful. T-strings -- which begin with ``t`` or ``T`` -- share the
+  same syntax as f-strings but, unlike f-strings, produce a
+  :class:`~string.templatelib.Template` instance rather than a simple ``str``.
+  Templates give you access to the static and interpolated (in curly braces)
+  parts of the string *before* they are combined into a final string.
+
+  ::
+
+     >>> name = "World"
+     >>> template = t"Hello {name}!"
+     >>> template.strings
+     ('Hello ', '!')
+     >>> template.values
+     ('World',)
+
 * The :meth:`str.format` method of strings requires more manual
   effort.  You'll still use ``{`` and ``}`` to mark where a variable
   will be substituted and can provide detailed formatting directives,
   but you'll also need to provide the information to be formatted. In the following code
   block there are two examples of how to format variables:
 
-
   ::
 
      >>> yes_votes = 42_572_654
@@ -95,10 +110,11 @@ Some examples::
    >>> repr((x, y, ('spam', 'eggs')))
    "(32.5, 40000, ('spam', 'eggs'))"
 
-The :mod:`string` module contains a :class:`~string.Template` class that offers
-yet another way to substitute values into strings, using placeholders like
-``$x`` and replacing them with values from a dictionary, but offers much less
-control of the formatting.
+The :mod:`string` module also contains support for so-called
+:ref:`$-strings <template-strings-pep292>` that offer yet another way to
+substitute values into strings, using placeholders like ``$x`` and replacing
+them with values from a dictionary. This syntax is easy to use, although
+it offers much less control of the formatting.
 
 .. index::
    single: formatted string literal
@@ -160,6 +176,186 @@ See :ref:`self-documenting expressions <bpo-36817-whatsnew>` for more informatio
 on the ``=`` specifier. For a reference on these format specifications, see
 the reference guide for the :ref:`formatspec`.
 
+
+.. _tut-t-strings:
+
+Template String Literals
+-------------------------
+
+:ref:`Template string literals <t-strings>` (also called t-strings for short)
+are an extension of :ref:`f-strings <tut-f-strings>` that let you access the
+static and interpolated parts of a string *before* they are combined into a
+final string. This provides for greater control over how the string is
+formatted.
+
+The most common way to create a :class:`~string.templatelib.Template` instance
+is to use the :ref:`t-string literal syntax <t-strings>`. This syntax is
+identical to that of :ref:`f-strings` except that it uses a ``t`` instead of
+an ``f``:
+
+     >>> name = "World"
+     >>> template = t"Hello {name}!"
+     >>> template.strings
+     ('Hello ', '!')
+     >>> template.values
+     ('World',)
+
+:class:`~!string.templatelib.Template` instances are iterable, yielding each
+string and :class:`~string.templatelib.Interpolation` in order:
+
+.. testsetup::
+
+   name = "World"
+   template = t"Hello {name}!"
+
+.. doctest::
+
+     >>> list(template)
+     ['Hello ', Interpolation('World', 'name', None, ''), '!']
+
+Interpolations represent expressions inside a t-string. They contain the
+evaluated value of the expression (``'World'`` in this example), the text of
+the original expression (``'name'``), and optional conversion and format
+specification attributes.
+
+Templates can be processed in a variety of ways. For instance, here's code that
+converts static strings to lowercase and interpolated values to uppercase:
+
+     >>> from string.templatelib import Template
+     >>>
+     >>> def lower_upper(template: Template) -> str:
+     ...     return ''.join(
+     ...         part.lower() if isinstance(part, str) else part.value.upper()
+     ...         for part in template
+     ...     )
+     ...
+     >>> name = "World"
+     >>> template = t"Hello {name}!"
+     >>> lower_upper(template)
+     'hello WORLD!'
+
+Template strings are particularly useful for sanitizing user input. Imagine
+we're building a web application that has user profile pages. Perhaps the
+``User`` class is defined like this:
+
+     >>> from dataclasses import dataclass
+     >>>
+     >>> @dataclass
+     ... class User:
+     ...     name: str
+     ...
+
+Imagine using f-strings in to generate HTML for the ``User``:
+
+.. testsetup::
+
+     class User:
+         name: str
+         def __init__(self, name: str):
+             self.name = name
+
+
+.. doctest::
+
+     >>> # Warning: this is dangerous code. Don't do this!
+     >>> def user_html(user: User) -> str:
+     ...     return f"<div><h1>{user.name}</h1></div>"
+     ...
+
+This code is dangerous because our website lets users type in their own names.
+If a user types in a name like ``"<script>alert('evil');</script>"``, the
+browser will execute that script when someone else visits their profile page.
+This is called a *cross-site scripting (XSS) vulnerability*, and it is a form
+of *injection vulnerability*. Injection vulnerabilities occur when user input
+is included in a program without proper sanitization, allowing malicious code
+to be executed. The same sorts of vulnerabilities can occur when user input is
+included in SQL queries, command lines, or other contexts where the input is
+interpreted as code.
+
+To prevent this, instead of using f-strings, we can use t-strings. Let's
+update our ``user_html()`` function to return a :class:`~string.templatelib.Template`:
+
+     >>> from string.templatelib import Template
+     >>>
+     >>> def user_html(user: User) -> Template:
+     ...     return t"<div><h1>{user.name}</h1></div>"
+
+Now let's implement a function that sanitizes *any* HTML
+:class:`~!string.templatelib.Template`:
+
+     >>> from html import escape
+     >>> from string.templatelib import Template
+     >>>
+     >>> def sanitize_html_template(template: Template) -> str:
+     ...     return ''.join(
+     ...         part if isinstance(part, str) else escape(part.value)
+     ...         for part in template
+     ...     )
+     ...
+
+This function iterates over the parts of the
+:class:`~!string.templatelib.Template`, escaping any interpolated values using
+the :func:`html.escape` function, which converts special characters like `<`,
+`>`, and `&` into their HTML-safe equivalents.
+
+Now we can tie it all together:
+
+.. testsetup::
+
+     from dataclasses import dataclass
+     from string.templatelib import Template
+     from html import escape
+     @dataclass
+     class User:
+         name: str
+     def sanitize_html_template(template: Template) -> str:
+         return ''.join(
+             part if isinstance(part, str) else escape(part.value)
+             for part in template
+         )
+     def user_html(user: User) -> Template:
+         return t"<div><h1>{user.name}</h1></div>"
+
+.. doctest::
+
+     >>> evil_user = User(name="<script>alert('evil');</script>")
+     >>> template = user_html(evil_user)
+     >>> safe = sanitize_html_template(template)
+     >>> print(safe)
+     <div><h1>&lt;script&gt;alert(&#x27;evil&#x27;);&lt;/script&gt;</h1></div>
+
+We are no longer vulnerable to XSS attacks because we are escaping the
+interpolated values before they are included in the rendered HTML.
+
+Of course, there's no need for code that processes
+:class:`~!string.templatelib.Template` instances to be limited to returning a
+simple string. For instance, we could imagine defining a more complex ``html()``
+function that returns a structured representation of the HTML:
+
+     >>> from dataclasses import dataclass
+     >>> from string.templatelib import Template
+     >>> from html.parser import HTMLParser
+     >>>
+     >>> @dataclass
+     ... class Element:
+     ...     tag: str
+     ...     attributes: dict[str, str]
+     ...     children: list[str | Element]
+     ...
+     >>> def parse_html(template: Template) -> Element:
+     ...     """
+     ...     Uses python's built-in HTMLParser to parse the template,
+     ...     handle any interpolated values, and return a tree of
+     ...     Element instances.
+     ...     """
+     ...     pass
+     ...
+
+A full implementation of this function would be quite complex and is not
+provided here. That said, the fact that it is possible to implement a method
+like ``parse_html()`` showcases the flexibility and power of t-strings.
+
+
 .. _tut-string-format:
 
 The String format() Method