VTXWriter: support file modes

[schnellerhase]

Adds support to ADIOS2Writer and VTXWriter to open files in append mode (following interface of VTKFile). Enabling usage such as

import dolfinx
from mpi4py import MPI

mesh = dolfinx.mesh.create_unit_square(comm := MPI.COMM_WORLD, n := 10, n)
V = dolfinx.fem.functionspace(mesh, ("CG", 1))
f = dolfinx.fem.Function(V)

# this usage previously only held the 9th iterate in the output, i.e. was not supported
for i in range(10):
    f.x.array[:] = i
    with dolfinx.io.VTXWriter(
        comm,
        "out_a.bp",
        "a",
        [f],
    ) as writer:
        writer.write(i)

# now holds equivalent output to:
with dolfinx.io.VTXWriter(
    comm,
    "out_w.bp",
    "w",
    [f],
) as writer:
    for i in range(10):
        f.x.array[:] = i
        writer.write(i)

Demonstrates the new feature, with simplified I/o logic in the naiver stokes demo.

[jhale]

else if and else?

[jhale]

Did you consider just wrapping adios2::Mode directly as an enum?

[schnellerhase]

The ADIOS2Writer operates directly on adios::Mode only the VTXWriter on the string. I understood the second as being the no longer adios2 interface specific implementation of dolfinx's interface, therefore not using the enum there.

[jhale]

Leaving aside Enum vs no Enum, it's generally most transparent to map modes directly e.g. "Read" -> "adios2::Mode::Read". Then I have a clear idea what "Read" means in the context of the underlying library.

[schnellerhase]

r, w are the unix names for the operation and also what we use for the VTKFile interface.

[jorgensd]

I'm a bit sceptical about adding an "append" mode, as the VTXScheme shouldn't (or at least couldn't previously) be modified at runtime with ADIOS2, which is why the initializer takes in the functions/meshes to store at each time-step, instead of following the XDMF file version where we write when we can.

adding an append-mode needs us to revise the internal logic to ensure for consistent input at multiple steps.
We would also have to thoroughly check which variables ADIOS2 now allows to be time-dependent (last section of https://adios2.readthedocs.io/en/latest/ecosystem/visualization.html)

[schnellerhase]

I'm a bit sceptical about adding an "append" mode, as the VTXScheme shouldn't (or at least couldn't previously) be modified at runtime with ADIOS2, which is why the initializer takes in the functions/meshes to store at each time-step, instead of following the XDMF file version where we write when we can.

adding an append-mode needs us to revise the internal logic to ensure for consistent input at multiple steps. We would also have to thoroughly check which variables ADIOS2 now allows to be time-dependent (last section of https://adios2.readthedocs.io/en/latest/ecosystem/visualization.html)

I checked with the following script, which seems to correctly append to the xml contents (while currently duplicating everything though) of the VTXScheme. That should allow us to enable append mode writing in principle, right?

import dolfinx
from mpi4py import MPI

mesh = dolfinx.mesh.create_unit_square(comm := MPI.COMM_WORLD, n := 10, n)
V = dolfinx.fem.functionspace(mesh, ("CG", 1))
V2 = dolfinx.fem.functionspace(mesh, ("CG", 2))
f = dolfinx.fem.Function(V)
f.x.array[:] = 1
f2 = dolfinx.fem.Function(V2)
f2.x.array[:] = 2

with dolfinx.io.VTXWriter(
    comm,
    "out.bp",
    "w",
    [f],
) as writer:
    writer.write(0)


with dolfinx.io.VTXWriter(
    comm,
    "out.bp",
    "a",
    [f2],
) as writer:
    writer.write(1)

from adios2 import Stream

with Stream("out.bp", "r") as s:
    # steps comes from the stream
    for _ in s.steps():
        # track current step
        print(f"Current step is {s.current_step()}")

        # inspect variables in current step
        # for name, info in s.available_variables().items():
        #     print("variable_name: " + name, end=" ")
        #     for key, value in info.items():
        #         print("\t" + key + ": " + value, end=" ")
        #     print()

        xml = s.read_attribute("vtk.xml")
        print(xml)

Output:

Current step is 0
<?xml version="1.0"?>
<VTKFile type="UnstructuredGrid" version="0.1">
  <UnstructuredGrid>
    <Piece NumberOfPoints="NumberOfNodes" NumberOfCells="NumberOfCells">
      <Points>
        <DataArray Name="geometry" />
      </Points>
      <Cells>
        <DataArray Name="connectivity" />
        <DataArray Name="types" />
      </Cells>
      <PointData>
        <DataArray Name="TIME">step</DataArray>
        <DataArray Name="vtkOriginalPointIds" />
        <DataArray Name="vtkGhostType" />
        <DataArray Name="f" />
      </PointData>
    </Piece>
  </UnstructuredGrid>
</VTKFile>

Current step is 1
<?xml version="1.0"?>
<VTKFile type="UnstructuredGrid" version="0.1">
  <UnstructuredGrid>
    <Piece NumberOfPoints="NumberOfNodes" NumberOfCells="NumberOfCells">
      <Points>
        <DataArray Name="geometry" />
      </Points>
      <Cells>
        <DataArray Name="connectivity" />
        <DataArray Name="types" />
      </Cells>
      <PointData>
        <DataArray Name="TIME">step</DataArray>
        <DataArray Name="vtkOriginalPointIds" />
        <DataArray Name="vtkGhostType" />
        <DataArray Name="f" />
      </PointData>
    </Piece>
  </UnstructuredGrid>
</VTKFile>

[jorgensd]

@schnellerhase They did change how attributes work in adios2 a while ago (ornladios/ADIOS2#4471) which blurred the lines between variables and attributes, which I guess it makes it work.