Reporting IO error

[kcho]

While running harmonization for AMP-SCZ dMRI data, I've encountered IO error raised at a simple print statement, right after the ANTs template creation step from harmonization.py, suggesting that sys.stdout has been closed earlier in execution.

The IO error was replicated in the following methods of executing multi-shell-dMRIharmonization code.

• using pnlpip3 environment on dna007
• using singularity container on dna007
• using singularity container on a workstation at PNL

See the error message below.

--------------------------------------------------------------------------------------
 Done creating: /data/predict2/MRI_ROOT/derivatives/dMRIharmonization/ME_device_1_soft_1_SG_device_1_soft_1/template/template0.nii.gz /data/predict2/MRI_ROOT/derivati
ves/dMRIharmonization/ME_device_1_soft_1_SG_device_1_soft_1/template/template1.nii.gz
 Script executed in 93343 seconds
 25h 55m 43s
--------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/pnl/soft/pnlpipe3/multi-shell-dMRIharmonization/lib/harmonization.py", line 529, in <module>
    pipeline.run()
  File "/data/pnl/soft/pnlpipe3/miniconda3/envs/pnlpipe3/lib/python3.6/site-packages/plumbum/cli/application.py", line 572, in run
    retcode = inst.main(*tailargs)
  File "/data/pnl/soft/pnlpipe3/multi-shell-dMRIharmonization/lib/harmonization.py", line 509, in main
    self.createTemplate()
  File "/data/pnl/soft/pnlpipe3/multi-shell-dMRIharmonization/lib/harmonization.py", line 215, in createTemplate
    print('calculating dti statistics i.e. mean, std calculation for reference site')
ValueError: I/O operation on closed file.
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/data/pnl/soft/pnlpipe3/miniconda3/envs/pnlpipe3/lib/python3.6/site-packages/plumbum/colorlib/styles.py", line 511, in now
    self.stdout.write(str(self))
ValueError: I/O operation on closed file.
Traceback (most recent call last):
  File "/data/pnl/soft/pnlpipe3/multi-shell-dMRIharmonization/lib/multi-shell-harmonization.py", line 227, in <module>
    multi_shell_pipeline.run()
  File "/data/pnl/soft/pnlpipe3/miniconda3/envs/pnlpipe3/lib/python3.6/site-packages/plumbum/cli/application.py", line 572, in run
    retcode = inst.main(*tailargs)
  File "/data/pnl/soft/pnlpipe3/multi-shell-dMRIharmonization/lib/multi-shell-harmonization.py", line 192, in main
    '--create'] + pipeline_vars), shell= True)
  File "/data/pnl/soft/pnlpipe3/miniconda3/envs/pnlpipe3/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '/data/pnl/soft/pnlpipe3/multi-shell-dMRIharmonization/lib/harmonization.py --tar_list /data/predict2/MRI_ROOT/derivatives/dMRI
harmonization/ME_device_1_soft_1_SG_device_1_soft_1/harmonization_input_target_b3000.csv --bshell_b 3000 --ref_list /data/predict2/MRI_ROOT/derivatives/dMRIharmonizat
ion/ME_device_1_soft_1_SG_device_1_soft_1/harmonization_input_ref_b3000.csv --create --tar_name SG_device_1_soft_1 --nshm -1 --nproc 36 --template /data/predict2/MRI_
ROOT/derivatives/dMRIharmonization/ME_device_1_soft_1_SG_device_1_soft_1/template --ref_name ME_device_1_soft_1 --nzero 10 --debug --verbose' returned non-zero exit s
tatus 1.

The issue appears to stem from how subprocess.check_call is invoked together with stdout/stderr handling and file closing logic.

Current pattern (simplified):

subprocess.check_call(
    cmd,
    shell=True,
    stdout=f,
    stderr=sys.stdout
)

Problems with this pattern:

1. stderr=sys.stdout creates a hard dependency on sys.stdout remaining open.
2. In some environments (HPC, wrappers, logging frameworks), f may be a wrapper around stdout.
3. Closing f can indirectly close the underlying stdout.
4. Once stdout is closed, any later print() or atexit logging crashes with
   ValueError: I/O operation on closed file.

This explains why the failure happens after the subprocess finishes and often only near program exit.

Minimal reproduction

import sys, atexit

def goodbye():
    print("atexit printing")

atexit.register(goodbye)
sys.stdout.close()

This produces the same error signature.

Proposed fix

I'll create a PR for the team's review with the following change in lib/buildTemplate.py and lib/reconstSignal.py

-    if f.name!='<sys.stdout>':
+    if f is not sys.stdout:
         f.close()