Harden header/host control-char validation: unicode values + trailing newline

Two defense-in-depth gaps in the 2.7.18.14 header-injection backports
(CVE-2024-6923 / control-char cluster), surfaced in review of PR #85:

- wsgiref.headers._check_string only checked `str`, so a `unicode` header
  name/value carrying control characters bypassed the guard and could still
  be serialized. Check `basestring` instead.

- email.generator.NEWLINE_WITHOUT_FWSP used consuming character classes
  ([^ \t]) that require a following character, so a value ending in a bare
  CR/LF/CRLF was not rejected -- the generator then appends its own newline,
  prematurely terminating the header block. Add an end-of-string anchor so a
  trailing bare CR/LF is caught too, and drop the redundant `\r\n` branch (a
  CRLF that is not a valid fold is already matched by the lone-LF branch or
  the end anchor) -- per PR #86 review. Valid CRLF folding is still permitted.

Regression tests extended in test_wsgiref (unicode names/values) and
test_email (trailing-newline values). A third review comment, about
urlparse._check_bracketed_host leaking TypeError from socket.inet_pton on
unicode hosts, does not reproduce: CPython 2.7's "s" arg parser encodes
unicode, so invalid hosts raise UnicodeEncodeError (a ValueError subclass)
or socket.error, both already handled.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: icanhasmath

Lib/email/generator.py

@@ -16,9 +16,11 @@
 from email.errors import HeaderWriteError
 
 # Matches a CR/LF that is NOT part of a valid header folding (i.e. not
-# immediately followed by folding whitespace).  Used to detect injected
-# newlines in generated headers (CVE-2024-6923).
-NEWLINE_WITHOUT_FWSP = re.compile(r'\r\n[^ \t]|\r[^ \n\t]|\n[^ \t]')
+# immediately followed by folding whitespace), including a bare CR/LF at the
+# end of the string.  Used to detect injected newlines in generated headers
+# (CVE-2024-6923).  A '\r\n' need not be matched on its own: a CRLF that isn't
+# a valid fold is already caught by the lone-LF branch or the end anchor.
+NEWLINE_WITHOUT_FWSP = re.compile(r'\r[^ \n\t]|\n[^ \t]|[\r\n]$')
 
 UNDERSCORE = '_'
 NL = '\n'

Lib/email/test/test_email.py

@@ -87,7 +87,13 @@ def test_string_rejects_header_injection(self):
         from cStringIO import StringIO
         for bad in ('value\r\nInjected: header',
                     'value\nInjected: header',
-                    'value\rstuff'):
+                    'value\rstuff',
+                    # A bare CR/LF/CRLF at end-of-string is also an injected
+                    # newline: the generator appends its own newline after it,
+                    # prematurely terminating the header block.
+                    'value\n',
+                    'value\r',
+                    'value\r\n'):
             msg = Message()
             msg['Subject'] = bad
             g = Generator(StringIO(), maxheaderlen=0)

Lib/test/test_wsgiref.py

@@ -309,8 +309,14 @@ def testControlCharactersRejected(self):
         self.assertRaises(ValueError, h.add_header, 'Foo', 'a\nb')
         self.assertRaises(ValueError, h.add_header, 'Foo', 'ok', baz='x\ry')
         self.assertRaises(ValueError, Headers, [('Foo', 'a\nb')])
+        # unicode names/values must be rejected too (a str-only check would
+        # let a unicode value carrying control characters slip through).
+        self.assertRaises(ValueError, h.__setitem__, 'Foo', u'bar\r\nInjected: 1')
+        self.assertRaises(ValueError, h.__setitem__, u'Ba\nd', 'value')
+        self.assertRaises(ValueError, Headers, [(u'Foo', u'a\nb')])
         # Benign headers still work.
         h['Foo'] = 'bar'
+        h[u'Bar'] = u'baz'
         h.add_header('Content-Disposition', 'attachment', filename='ok.txt')
 
     def testMappingInterface(self):

Lib/wsgiref/headers.py

@@ -18,7 +18,10 @@
 
 def _check_string(value):
     """Reject header names/values containing control characters."""
-    if isinstance(value, str) and _control_chars_re.search(value):
+    # Check both str and unicode: in Python 2 a unicode header value would
+    # otherwise bypass the guard and could still be serialized, leaving the
+    # response-splitting protection incomplete.
+    if isinstance(value, basestring) and _control_chars_re.search(value):
         raise ValueError("Control characters not allowed in headers")
     return value