Use ObsBuilder scripts from SPOC in bufr conversion

[mranst]

Description

#773

I've been experimenting a bit with using the spoc scripts in the bufr-to-ioda conversion to see if I could get it work. I believe it mostly works, but it is a somewhat crude implementation, so it will probably require some fine-tuning from people more familiar with obs handling

This PR adds the CloneSpoc task, which can grab a branch set by spoc_tag in experiment.yaml, or point to an existing spoc_source_directory. Bufr2Ioda then uses the scripts to run the conversion.

A few questions/notes:

• It seems like the scripts are expecting the corresponding yamls to live in the same directory they are being executed from? I may be misunderstanding something, but i added this process to CloneSpoc to accomodate:
  

  swell/src/swell/tasks/clone_spoc.py

  Lines 66 to 71 in eca7d5c

  	# Copy the mapping yaml's to the script path
  	# ------------------------------------------
  	config_glob = os.path.join(spoc_exp_path, 'dump', 'config', 'atmosphere', '*yaml')
  	script_path = os.path.join(spoc_exp_path, 'dump', 'scripts', 'atmosphere')
  	for config_file in glob.glob(config_glob):
  	shutil.copy(config_file, script_path)

• The output ioda files seem to be combined with all satId's in one file, not sure how significant this is

• There are some obs types that have separate yamls under the bufr2netcdf.x schema, that don't have individual python scripts (I think this is noted in Incorporating SPOC into swell convert_bufr suite #773)

[cohen-seth]

A few questions/notes:

1. "It seems like the scripts are expecting the corresponding yamls to live in the same directory they are being executed from? I may be misunderstanding something, but i added this process to CloneSpoc to accomodate:"

• This is correct the scripts are expecting the corresponding yamls to live in the same directory they are being executed from. This was done to preserve the yaml used for the conversion within the run directory for referring to later & for validation; that way, when we add the capability to customize the yamls and/or when spoc yamls are updated there will be a way of knowing how the files were converted. (See directory tree below)
• CloneSpoc will be very helpful. Previously, I had to manually add the spoc yamls to the configuration/jedi/bufr2ioda/bufr2netcdf_x/ directory - should be able to remove these going forward.

run/20231010T000000Z/geos_atmosphere/
│
├── bufr/
├── ioda/
├── spoc_radiance_1bamua.yaml
├── spoc_radiance_1bmhs.yaml
├── spoc_radiance_atms.yaml
├── spoc_radiance_avcsam.yaml
├── spoc_radiance_avcspm.yaml
├── spoc_radiance_gpsro.yaml
└── spoc_radiance_mtiasi.yaml

2. "The output ioda files seem to be combined with all satId's in one file, not sure how significant this is"

• I believe we want them in separate files. Double check with @rtodling
• this functionality of bufr2netcd.x is still being developed. There are 2 methods, one method that only partially works and one method that requires more steps. So I had to use the 2nd method, the --no-gather method. We will want to switch to the other method once it's functional.

[cohen-seth]

Are these obs builder python scripts placeholders?