Ildg 1.2 by gray95 · Pull Request #15 · telos-collaboration/Grid

gray95 · 2025-12-03T15:15:52Z

added ability to write ILDG 1.2 spec compliant lattice gauge fields (SU(N) and Sp(2N))
added ability to read ILDG 1.2 spec fields while retaining backwards compatibility for older fields
added ability to write reduced format SU and Sp fields
added ability to read reduced format SU and Sp fields (SU N=2,3 only currently)
added Test_ildg_reducedfmt_io.cc to tests/IO which checks the various read/write options work.

Much of the logic for writing fields is worked out in IldgWriter::writeConfiguration at compile time via template arguments and if constexprs. group_name, matrix_fmt, and fp_fmt are the relevant template parameters.
The logic for reading fields in IldgReader::readConfiguration has to be done at runtime. Extra intermediate datatypes for Sp(N) fields are specified in parallelIO/Metadata.h

edbennett

Hi Gaurav

Many thanks for your work on this so far, I think we're almost there.

I've requested some changes in the attached review. Most of them are pretty trivial, relating to not wanting to pollute the Git logs. (Upstream has got annoyed at other people doing this before, and I want us to stay on their good side.)

One thing I haven't highlighted specific instances of: where you add new functionality (either new functions, or significant edits to existing ones), please can you add detailed comments describing what you've done, linking to reference material where relevant? I realise this isn't standard practice in Grid, but our lives would have been a lot easier if it were, and so I'm aiming for our new contributions to be better about this to make life easier for the next person.

For example, reconstructSp should have a 5–10 line comment above it describing what the expected block structure of a symplectic matrix is, and hence what A and B in the comments mean, and have a link to the relevant section of the ILDG binary spec 1.2.

edbennett · 2025-12-04T22:51:45Z

@@ -107,2 +99,2 @@
 public:
  constexpr static const char* const Name = "Binary";


This change and the next few aren't related to this PR; in principle you could rebase your branch onto the current telos-collaboration:develop to get GitHub to remove them.

edbennett · 2025-12-05T15:03:11Z

P.S. one thing I meant to add in the message yesterday: please make sure to update the documentation in documentation/manual.rst, where the ILDG checkpointers are explicitly mentioned.

…ge after ildg_binary_data

…teConfiguration now has template args specifying the gauge group and whether to write to reduced storage format. Started working on a reduced storage test.

…ingle precision writing

…ow the default is to write to double (with no group reduction). added Test_ildg_reducedfmt_io for testing writing/reading to reduced group format and single/double precision.

…fault.

…ore xmas

…plated on the gauge group.

…ed format Test

…r end-to-end test.

…Sp differentiation

…ding to indentation issues

…y explain the <rows/> element logic when reading ildg lattices.

…inting with reduced format su/sp lattices

…ckwards compatibility. this necessitates explicitly specifying an additional template param in the ildg reduced format test.

…unit test

…emove little endian from initialisation check

… checkpointing

merge feature - hmc checkpointing with ildg reduced format read/write functionality.

edbennett · 2026-04-01T15:26:28Z

Many thanks. I tidied up a couple of loose ends.

I've just tested this on Tursa, and it doesn't build, giving errors like:

../../Grid/libGrid.a(Init.o): In function `Grid::is_perm_even(std::vector<int, std::allocator<int> >&)':
/mnt/lustre/tursafs1/home/dp208/dp208/shared/src/Grid_20260401_gsr/Grid/parallelIO/MetaData.h:256: multiple definition of `Grid::is_perm_even(std::vector<int, std::allocator<int> >&)'
Test_hmc_Sp_ildg_pureGaugeWilson.o:/mnt/lustre/tursafs1/home/dp208/dp208/shared/src/Grid_20260401_gsr/Grid/parallelIO/MetaData.h:256: first defined here
../../Grid/libGrid.a(FlightRecorder.o): In function `Grid::is_perm_even(std::vector<int, std::allocator<int> >&)':
/mnt/lustre/tursafs1/home/dp208/dp208/shared/src/Grid_20260401_gsr/Grid/parallelIO/MetaData.h:256: multiple definition of `Grid::is_perm_even(std::vector<int, std::allocator<int> >&)'
Test_hmc_Sp_ildg_pureGaugeWilson.o:/mnt/lustre/tursafs1/home/dp208/dp208/shared/src/Grid_20260401_gsr/Grid/parallelIO/MetaData.h:256: first defined here
../../Grid/libGrid.a(StaggeredKernelsInstantiation.o): In function `cudaLaunchKernel<char>':
/mnt/lustre/tursafs1/home/dp208/dp208/shared/src/Grid_20260401_gsr/Grid/parallelIO/MetaData.h:256: multiple definition of `Grid::is_perm_even(std::vector<int, std::allocator<int> >&)'
Test_hmc_Sp_ildg_pureGaugeWilson.o:/mnt/lustre/tursafs1/home/dp208/dp208/shared/src/Grid_20260401_gsr/Grid/parallelIO/MetaData.h:256: first defined here

configure line was:

../configure --disable-fermion-instantiations --prefix /home/dp208/dp208/shared/prefixes/prefix_grid_20260224 --with-gmp=/home/dp208/dp208/dc-benn2/prefix-gmp --with-mpfr=/home/dp208/dp208/dc-benn2/prefix-gmp --with-lime=/home/dp208/dp208/dc-benn2/prefix --enable-comms=mpi --enable-simd=GPU --enable-shm=nvlink --enable-gen-simd-width=64 --enable-accelerator=cuda --enable-Sp --enable-Nc=4 --disable-unified CXX=nvcc LDFLAGS="-cudart shared -lcublas" CXXFLAGS="-ccbin mpicxx -gencode arch=compute_80,code=sm_80 -std=c++17 -cudart shared"

Have you been able to successfully test on Tursa?

gray95 · 2026-04-01T15:45:53Z

Many thanks. I tidied up a couple of loose ends.

I've just tested this on Tursa, and it doesn't build, giving errors like:

../../Grid/libGrid.a(Init.o): In function `Grid::is_perm_even(std::vector<int, std::allocator<int> >&)':
/mnt/lustre/tursafs1/home/dp208/dp208/shared/src/Grid_20260401_gsr/Grid/parallelIO/MetaData.h:256: multiple definition of `Grid::is_perm_even(std::vector<int, std::allocator<int> >&)'
Test_hmc_Sp_ildg_pureGaugeWilson.o:/mnt/lustre/tursafs1/home/dp208/dp208/shared/src/Grid_20260401_gsr/Grid/parallelIO/MetaData.h:256: first defined here
../../Grid/libGrid.a(FlightRecorder.o): In function `Grid::is_perm_even(std::vector<int, std::allocator<int> >&)':
/mnt/lustre/tursafs1/home/dp208/dp208/shared/src/Grid_20260401_gsr/Grid/parallelIO/MetaData.h:256: multiple definition of `Grid::is_perm_even(std::vector<int, std::allocator<int> >&)'
Test_hmc_Sp_ildg_pureGaugeWilson.o:/mnt/lustre/tursafs1/home/dp208/dp208/shared/src/Grid_20260401_gsr/Grid/parallelIO/MetaData.h:256: first defined here
../../Grid/libGrid.a(StaggeredKernelsInstantiation.o): In function `cudaLaunchKernel<char>':
/mnt/lustre/tursafs1/home/dp208/dp208/shared/src/Grid_20260401_gsr/Grid/parallelIO/MetaData.h:256: multiple definition of `Grid::is_perm_even(std::vector<int, std::allocator<int> >&)'
Test_hmc_Sp_ildg_pureGaugeWilson.o:/mnt/lustre/tursafs1/home/dp208/dp208/shared/src/Grid_20260401_gsr/Grid/parallelIO/MetaData.h:256: first defined here

configure line was:

../configure --disable-fermion-instantiations --prefix /home/dp208/dp208/shared/prefixes/prefix_grid_20260224 --with-gmp=/home/dp208/dp208/dc-benn2/prefix-gmp --with-mpfr=/home/dp208/dp208/dc-benn2/prefix-gmp --with-lime=/home/dp208/dp208/dc-benn2/prefix --enable-comms=mpi --enable-simd=GPU --enable-shm=nvlink --enable-gen-simd-width=64 --enable-accelerator=cuda --enable-Sp --enable-Nc=4 --disable-unified CXX=nvcc LDFLAGS="-cudart shared -lcublas" CXXFLAGS="-ccbin mpicxx -gencode arch=compute_80,code=sm_80 -std=c++17 -cudart shared"

Have you been able to successfully test on Tursa?

huh, i will build on tursa and take a look at this. I have tested on sunbird only so far.

gray95 · 2026-04-01T20:20:45Z

that should fix the tursa compilation issue. I have compiled and successfully run Test_io_mungers, Test_ildg_reducedfmt_io and Test_hmc_Sp_ildg_pureGaugeWilson. you can see some of the resulting slurm output in /home/dp208/dp208/dc-ray2/grid-ildg.

edbennett

lgtm

edbennett requested changes Dec 4, 2025

View reviewed changes

gray95 force-pushed the ildg-1.2 branch from 2430dd7 to 20de41b Compare January 21, 2026 18:40

gray95 force-pushed the ildg-1.2 branch from 97c8dec to 5abd1b3 Compare February 18, 2026 18:02

gray95 added 26 commits February 25, 2026 16:48

add <rows/> element to ildg_format and put ildg_lfn in it's own messa…

35f05e8

…ge after ildg_binary_data

readConfiguration is backwards compatible with ildg<1.2 lattices. wri…

a73b6bc

…teConfiguration now has template args specifying the gauge group and whether to write to reduced storage format. Started working on a reduced storage test.

sp mungers written and tested and writeConfiguration can now handle s…

3e7eb0e

…ingle precision writing

writeConfiguration correctly writes out precision into ildg header. n…

d5e51c8

…ow the default is to write to double (with no group reduction). added Test_ildg_reducedfmt_io for testing writing/reading to reduced group format and single/double precision.

using scoped enum to specify floating-point precision instead of boolean

38f3cd2

cleaned up some comments and set test_ildg_reducedfmt to use SU by de…

d0a7f3a

…fault.

started updating manual.rst and commit some trailing changes from bef…

f383295

…ore xmas

one more draft pr to go

a4357fa

added comments

71e3620

prepping for a git rebase

5adfc77

helper function to wrap BinaryIO::readLatticeObject

b67aa6d

Test.cc refactored with the core code in its own function that is tem…

cd3a6ff

…plated on the gauge group.

added/cleaned up some comments

050f923

small change to docs

cfedb49

added if-guards and another explanatory comment to the new ildg reduc…

f322125

…ed format Test

started adding unit tests for Ildg IO classes. corrected pre-amble fo…

245849f

…r end-to-end test.

added 1 unit test for Binary munger

3570f08

make comment on Spmunger clearer

9b3e1fc

added unit test for Sp(un)munger

8c2cae5

added unit test for 3x2 mungers

623f4ee

adds unit test for reconstruct3(LorentzColourMatrix &cm)

e710f8d

adds unit test for reconstructSp and adds if-constexpr guards for SU/…

3d72d3c

…Sp differentiation

resolve merge conflict from SU(N) branch

6f112f9

fixed reconstructSU and tested with N=3/4

facc2b3

implements GaugeSU(un)mungers

b8fdece

testing new reconstructSU function and GaugeSUmungers

48cc7bd

gray95 and others added 22 commits March 11, 2026 17:15

minor formatting changes

a53f6f8

minor formatting changes - got rid of inconsistent tabs that were lea…

59ba80c

…ding to indentation issues

removes redundant ' == true' after std::is_same_v<>

ad959fd

changes unit test return type to void

9f02b0d

minor formatting issues - tabs v spaces. added comment to more clearl…

62d7907

…y explain the <rows/> element logic when reading ildg lattices.

edits comments and var names for clarity

22501e2

add assert statement to ensure either su or sp field read

f5d882c

augment CheckpointerParameters and ILDGCheckpointer to enable checkpo…

8a0cb02

…inting with reduced format su/sp lattices

streamline test of reduced sp2n hmc runner

348990c

adds default false value for CheckpointParameters.saveSmeared

246a070

fix check on gauge group from CheckpointerParameters.group

6d34f5f

change order of template parameters in read/writeConfiguration for ba…

ec440dc

…ckwards compatibility. this necessitates explicitly specifying an additional template param in the ildg reduced format test.

change order of template parameters in read/writeConfiguration for ba…

9da4067

…ckwards compatibility. this necessitates explicitly specifying an additional template param in the ildg reduced format test.

use Johnson-Trotter algorithm to ensure completeness of is_perm_even …

ce2c49c

…unit test

factor out logic for choosing IldgWriter template instantiation and r…

32b9cc3

…emove little endian from initialisation check

re-order for clarity and show unique_su parameter is available for su…

aa25bdf

… checkpointing

Merge branch 'sp2n-hmc-runner' into ildg-1.2

24eede8

merge feature - hmc checkpointing with ildg reduced format read/write functionality.

add runtime check for even Nc when writing Sp fields

1107e60

adds static_assert to catch users trying to write Sp fields with Nc odd

a7b1e47

revert change outside of target region

1a3e3a6

revert change outside target region

e354088

update copyright messages

a590585

mark is_perm_even as inline to avoid multiple re-definition errors

e608ff5

edbennett marked this pull request as ready for review April 2, 2026 23:24

edbennett approved these changes Apr 2, 2026

View reviewed changes

edbennett merged commit b07a948 into telos-collaboration:develop Apr 2, 2026
4 checks passed

gray95 deleted the ildg-1.2 branch April 8, 2026 22:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ildg 1.2#15

Ildg 1.2#15
edbennett merged 68 commits into
telos-collaboration:developfrom
gray95:ildg-1.2

gray95 commented Dec 3, 2025

Uh oh!

edbennett left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

edbennett Dec 4, 2025

Uh oh!

Uh oh!

Uh oh!

edbennett commented Dec 5, 2025

Uh oh!

edbennett commented Apr 1, 2026

Uh oh!

gray95 commented Apr 1, 2026

Uh oh!

gray95 commented Apr 1, 2026

Uh oh!

edbennett left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		@@ -107,2 +99,2 @@
		public:
		constexpr static const char* const Name = "Binary";

Conversation

gray95 commented Dec 3, 2025

Uh oh!

edbennett left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

edbennett Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

edbennett commented Dec 5, 2025

Uh oh!

edbennett commented Apr 1, 2026

Uh oh!

gray95 commented Apr 1, 2026

Uh oh!

gray95 commented Apr 1, 2026

Uh oh!

edbennett left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants