Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions Doc/library/argparse.rst
Original file line number Diff line number Diff line change
Expand Up @@ -767,9 +767,9 @@ how the command-line arguments should be handled. The supplied actions are:
Namespace(foo=42)

* ``'store_true'`` and ``'store_false'`` - These are special cases of
``'store_const'`` used for storing the values ``True`` and ``False``
respectively. In addition, they create default values of ``False`` and
``True`` respectively::
``'store_const'`` that respectively store the values ``True`` and ``False``
with default values of ``False`` and
``True``::

>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--foo', action='store_true')
Expand All @@ -789,8 +789,8 @@ how the command-line arguments should be handled. The supplied actions are:
>>> parser.parse_args('--foo 1 --foo 2'.split())
Namespace(foo=['0', '1', '2'])

* ``'append_const'`` - This stores a list, and appends the value specified by
the const_ keyword argument to the list; note that the const_ keyword
* ``'append_const'`` - This appends the value specified by
the const_ keyword argument to a list; note that the const_ keyword
argument defaults to ``None``. The ``'append_const'`` action is typically
useful when multiple arguments need to store constants to the same list. For
example::
Expand All @@ -801,8 +801,8 @@ how the command-line arguments should be handled. The supplied actions are:
>>> parser.parse_args('--str --int'.split())
Namespace(types=[<class 'str'>, <class 'int'>])

* ``'extend'`` - This stores a list and appends each item from the multi-value
argument list to it.
* ``'extend'`` - This appends each item from a multi-value
argument to a list.
The ``'extend'`` action is typically used with the nargs_ keyword argument
value ``'+'`` or ``'*'``.
Note that when nargs_ is ``None`` (the default) or ``'?'``, each
Expand All @@ -816,7 +816,7 @@ how the command-line arguments should be handled. The supplied actions are:

.. versionadded:: 3.8

* ``'count'`` - This counts the number of times a keyword argument occurs. For
* ``'count'`` - This counts the number of times an argument occurs. For
example, this is useful for increasing verbosity levels::

>>> parser = argparse.ArgumentParser()
Expand Down
8 changes: 8 additions & 0 deletions Doc/library/xml.sax.handler.rst
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,14 @@ for the feature and property names.

.. data:: feature_external_ges

.. warning::

Enabling opens a vulnerability to
`external entity attacks <https://en.wikipedia.org/wiki/XML_external_entity_attack>`_
if the parser is used with user-provided XML content.
Please reflect on your `threat model <https://en.wikipedia.org/wiki/Threat_model>`_
before enabling this feature.

| value: ``"http://xml.org/sax/features/external-general-entities"``
| true: Include all external general (text) entities.
| false: Do not include external general entities.
Expand Down
18 changes: 9 additions & 9 deletions Doc/library/zipfile.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ provides tools to create, read, write, append, and list a ZIP file. Any
advanced use of this module will require an understanding of the format, as
defined in `PKZIP Application Note`_.

This module does not currently handle multi-disk ZIP files.
This module does not handle multipart ZIP files.
It can handle ZIP files that use the ZIP64 extensions
(that is ZIP files that are more than 4 GiB in size). It supports
decryption of encrypted files in ZIP archives, but it currently cannot
decryption of encrypted files in ZIP archives, but it cannot
create an encrypted file. Decryption is extremely slow as it is
implemented in native Python rather than C.

Expand Down Expand Up @@ -175,7 +175,7 @@ The module defines the following items:

.. _zipfile-objects:

ZipFile Objects
ZipFile objects
---------------


Expand Down Expand Up @@ -248,7 +248,7 @@ ZipFile Objects
.. note::

*metadata_encoding* is an instance-wide setting for the ZipFile.
It is not currently possible to set this on a per-member basis.
It is not possible to set this on a per-member basis.

This attribute is a workaround for legacy implementations which produce
archives with names in the current locale encoding or code page (mostly
Expand Down Expand Up @@ -571,7 +571,7 @@ The following data attributes are also available:

.. _path-objects:

Path Objects
Path objects
------------

.. class:: Path(root, at='')
Expand Down Expand Up @@ -707,7 +707,7 @@ changes.

.. _pyzipfile-objects:

PyZipFile Objects
PyZipFile objects
-----------------

The :class:`PyZipFile` constructor takes the same parameters as the
Expand Down Expand Up @@ -784,7 +784,7 @@ The :class:`PyZipFile` constructor takes the same parameters as the

.. _zipinfo-objects:

ZipInfo Objects
ZipInfo objects
---------------

Instances of the :class:`ZipInfo` class are returned by the :meth:`.getinfo` and
Expand Down Expand Up @@ -954,7 +954,7 @@ Instances have the following methods and attributes:
.. _zipfile-commandline:
.. program:: zipfile

Command-Line Interface
Command-line interface
----------------------

The :mod:`zipfile` module provides a simple command-line interface to interact
Expand Down Expand Up @@ -1029,7 +1029,7 @@ From file itself
Decompression may fail due to incorrect password / CRC checksum / ZIP format or
unsupported compression method / decryption.

File System limitations
File system limitations
~~~~~~~~~~~~~~~~~~~~~~~

Exceeding limitations on different file systems can cause decompression failed.
Expand Down
23 changes: 21 additions & 2 deletions Lib/subprocess.py
Original file line number Diff line number Diff line change
Expand Up @@ -1613,6 +1613,10 @@ def _readerthread(self, fh, buffer):
fh.close()


def _writerthread(self, input):
self._stdin_write(input)


def _communicate(self, input, endtime, orig_timeout):
# Start reader threads feeding into a list hanging off of this
# object, unless they've already been started.
Expand All @@ -1631,8 +1635,23 @@ def _communicate(self, input, endtime, orig_timeout):
self.stderr_thread.daemon = True
self.stderr_thread.start()

if self.stdin:
self._stdin_write(input)
# Start writer thread to send input to stdin, unless already
# started. The thread writes input and closes stdin when done,
# or continues in the background on timeout.
if self.stdin and not hasattr(self, "_stdin_thread"):
self._stdin_thread = \
threading.Thread(target=self._writerthread,
args=(input,))
self._stdin_thread.daemon = True
self._stdin_thread.start()

# Wait for the writer thread, or time out. If we time out, the
# thread remains writing and the fd left open in case the user
# calls communicate again.
if hasattr(self, "_stdin_thread"):
self._stdin_thread.join(self._remaining_time(endtime))
if self._stdin_thread.is_alive():
raise TimeoutExpired(self.args, orig_timeout)

# Wait for the reader threads, or time out. If we time out, the
# threads remain reading and the fds left open in case the user
Expand Down
11 changes: 6 additions & 5 deletions Lib/test/_test_multiprocessing.py
Original file line number Diff line number Diff line change
Expand Up @@ -1204,7 +1204,7 @@ def test_put(self):
@classmethod
def _test_get(cls, queue, child_can_start, parent_can_continue):
child_can_start.wait()
#queue.put(1)
queue.put(1)
queue.put(2)
queue.put(3)
queue.put(4)
Expand All @@ -1229,15 +1229,16 @@ def test_get(self):
child_can_start.set()
parent_can_continue.wait()

time.sleep(DELTA)
for _ in support.sleeping_retry(support.SHORT_TIMEOUT):
if not queue_empty(queue):
break
self.assertEqual(queue_empty(queue), False)

# Hangs unexpectedly, remove for now
#self.assertEqual(queue.get(), 1)
self.assertEqual(queue.get_nowait(), 1)
self.assertEqual(queue.get(True, None), 2)
self.assertEqual(queue.get(True), 3)
self.assertEqual(queue.get(timeout=1), 4)
self.assertEqual(queue.get_nowait(), 5)
self.assertEqual(queue.get(), 5)

self.assertEqual(queue_empty(queue), True)

Expand Down
56 changes: 56 additions & 0 deletions Lib/test/test_subprocess.py
Original file line number Diff line number Diff line change
Expand Up @@ -1034,6 +1034,62 @@ def test_communicate_timeout_large_output(self):
(stdout, _) = p.communicate()
self.assertEqual(len(stdout), 4 * 64 * 1024)

def test_communicate_timeout_large_input(self):
# Test that timeout is enforced when writing large input to a
# slow-to-read subprocess, and that partial input is preserved
# for continuation after timeout (gh-141473).
#
# This is a regression test for Windows matching POSIX behavior.
# On POSIX, select() is used to multiplex I/O with timeout checking.
# On Windows, stdin writing must also honor the timeout rather than
# blocking indefinitely when the pipe buffer fills.

# Input larger than typical pipe buffer (4-64KB on Windows)
input_data = b"x" * (128 * 1024)

p = subprocess.Popen(
[sys.executable, "-c",
"import sys, time; "
"time.sleep(30); " # Don't read stdin for a long time
"sys.stdout.buffer.write(sys.stdin.buffer.read())"],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)

try:
timeout = 0.2
start = time.monotonic()
try:
p.communicate(input_data, timeout=timeout)
# If we get here without TimeoutExpired, the timeout was ignored
elapsed = time.monotonic() - start
self.fail(
f"TimeoutExpired not raised. communicate() completed in "
f"{elapsed:.2f}s, but subprocess sleeps for 30s. "
"Stdin writing blocked without enforcing timeout.")
except subprocess.TimeoutExpired:
elapsed = time.monotonic() - start

# Timeout should occur close to the specified timeout value,
# not after waiting for the subprocess to finish sleeping.
# Allow generous margin for slow CI, but must be well under
# the subprocess sleep time.
self.assertLess(elapsed, 5.0,
f"TimeoutExpired raised after {elapsed:.2f}s; expected ~{timeout}s. "
"Stdin writing blocked without checking timeout.")

# After timeout, continue communication. The remaining input
# should be sent and we should receive all data back.
stdout, stderr = p.communicate()

# Verify all input was eventually received by the subprocess
self.assertEqual(len(stdout), len(input_data),
f"Expected {len(input_data)} bytes output but got {len(stdout)}")
self.assertEqual(stdout, input_data)
finally:
p.kill()
p.wait()

# Test for the fd leak reported in http://bugs.python.org/issue2791.
def test_communicate_pipe_fd_leak(self):
for stdin_pipe in (False, True):
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
:mod:`xml.sax.handler`: Make Documentation of
:data:`xml.sax.handler.feature_external_ges` warn of opening up to `external
entity attacks <https://en.wikipedia.org/wiki/XML_external_entity_attack>`_.
Patch by Sebastian Pipping.
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Fix :func:`subprocess.Popen.communicate` timeout handling on Windows
when writing large input. Previously, the timeout was ignored during
stdin writing, causing the method to block indefinitely if the child
process did not consume input quickly. The stdin write is now performed
in a background thread, allowing the timeout to be properly enforced.
Loading