Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 36 additions & 9 deletions docs/misc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@ external library side (e.g. with ``torch.cuda.empty_cache``) before calling ASTR
get out-of-memory errors.


.. _set-single-gpu:

Choosing the GPU to use
-----------------------

Expand All @@ -54,12 +56,7 @@ with:

astra_mex('set_gpu_index', index);

**WARNING!** In `multithreading`_ contexts (Python), the GPU index will be set globally,
so it can't be used to reliably restrict a thread to a given GPU. Instead, you can avoid
calling ``astra.set_gpu_index`` whatsoever, and instead set the desired GPU with an
external library where the device context is thread-local, e.g. using
``torch.cuda.set_device``.

.. _set-multi-gpu:

Using several GPUs cooperatively
--------------------------------
Expand All @@ -82,11 +79,19 @@ computation. To do that, you can define the desired set of GPUs to be used:
functionality. For the rest, the first GPU in the specified index list will be used as
the fallback.

**WARNING!** In `multithreading`_ contexts (Python), the GPU indices will be
set globally (only) when specifying multiple indices, so it cannot be used to
reliably restrict a thread to a given set of GPUs. If this is required, you can
pass the set of GPUs as an option when creating the Astra projector or
algorithm object.



Multithreading
--------------

In Python, Astra supports concurrent execution of its algorithms in multiple threads.
In Python, Astra supports concurrent execution of its algorithms in multiple threads. This
is enabled by Astra dropping the Python global interpreter lock while executing an Astra algorithm.
This functionality is useful to process different inputs on different GPUs simultaneously.
Another use case is processing data blocks on the *same* GPU in parallel, which is useful
when a single ASTRA call under-utilizes the GPU. For instance, here is how one can
Expand Down Expand Up @@ -122,14 +127,36 @@ compute a fan-parallel projection using multithreading:

MATLAB doesn't support executing external libraries in multithreaded environments.

**WARNING!** No special care is taken about race conditions, so the user has to ensure that the
outputs of algorithms are not accessed simultaneously.
**WARNING!** No special care is taken about race conditions, so the user has to
ensure that the outputs of algorithms are not accessed before they are
finished, objects are created before they are used, and objects are not deleted
while still in use by other threads.

**WARNING!** MATLAB doesn't support executing external libraries in multithreaded
environments, so the only option is much less lightweight process-based concurrent
execution. The simplest approach is to just start several copies of MATLAB in batch
mode.

Python free threading
---------------------

Since Python 3.13, there is a separate Python interpreter that disables the global interpreter lock, called
`free threading <https://docs.python.org/3/howto/free-threading-python.html>`_.
Astra supports this mode, and most Astra functions are thread-safe.

There are two exceptions to the thread-safety of Astra calls: setting logging
parameters via the astra.log module is not thread-safe, and neither is
:ref:`passing multiple GPU indices to astra.set_gpu_index <set-multi-gpu>`.

On the other hand, using :ref:`astra.set_gpu_index with a single GPU index <set-single-gpu>` is thread-safe, and will set the active GPU only for the current
thread.

**WARNING!** No special care is taken about race conditions, so the user has to
ensure that the outputs of algorithms are not accessed before they are
finished, objects are created before they are used, and objects are not deleted
while still in use by other threads.


Masks
-----

Expand Down
Loading