Skip to content

Add support for central CLI and extra_config parameter configuration#629

Merged
Daniel-VM merged 33 commits into
BU-ISCIII:developfrom
Daniel-VM:develop
Jun 10, 2025
Merged

Add support for central CLI and extra_config parameter configuration#629
Daniel-VM merged 33 commits into
BU-ISCIII:developfrom
Daniel-VM:develop

Conversation

@Daniel-VM

@Daniel-VM Daniel-VM commented May 29, 2025

Copy link
Copy Markdown
Member

PR Checklist

  • This comment contains a description of changes (with reason).
  • Make sure your code lints (black and flake8).
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

PR description

This PR introduces a unified and consistent approach to argument handling across all modules in relecov-tools.

Main changes:

  • Centralized argument management: CLI arguments, extra config, and defaults are now merged with the following priority: CLI > extra_config > default.
  • All modules now use a consistent naming scheme for parameters, both in the CLI, the configuration file (initial_config.yaml), and the corresponding Python classes.
  • Added support for multiple CLI aliases for key parameters (e.g., --output-dir, --output_folder, etc.), but only the internal name (e.g., output_dir) is used in the config and code.
  • Updated all modules to use the new merging logic and consistent parameter names.
  • Improved robustness: empty string arguments are now converted to None.
  • Updated documentation, config templates, and linting.

Parameters renamed or standardized (by module):

1. Download Module (download_files)
Before: output_location
Now: output_dir
New aliases: --output-dir, --output_folder, --out-folder, --output_location, --output_path, --out_dir, --output

**2. Read Lab Metadata **
Before: metadata_out
Now: output_dir
New aliases as above

3. Validate
Before: out_folder
Now: output_dir
New aliases as above

4. send_report
Before: validate-file, receiver-email, email-psswd
Now: validate_file, receiver_email, email_psswd, additional_notes

5. Schema Mapping (map)
Before: json_data, output
Now: json_file, output_dir
New aliases for output

6. Upload to ENA (upload_to_ena)
Before: output_path
Now: output_dir
New aliases for output

7. Upload to GISAID (upload_to_gisaid)
Before: output_path
Now: output_dir
New aliases for output

8. Database Update (update_db)
Before: Arguments were passed positionally
Now: All arguments are passed as a dictionary using merge_with_extra_config (argument names are now consistent)

9. Read Bioinfo Metadata (read_bioinfo_metadata)
Before: out_dir
Now: output_dir
New aliases for output

10. Metadata Homogeneizer (metadata_homogeneizer)
Before: output
Now: output_dir
New aliases for output

11. Pipeline Manager (pipeline_manager)
Before: output
Now: output_dir
New aliases for output

12. Schema Builder (build_schema)
Before: out_dir
Now: output_dir
New aliases for output

13. Logs to Excel (logs_to_excel)
Before: output_folder
Now: output_dir
New aliases for output

14. Wrapper (wrapper)
Before: output_folder
Now: output_dir
New aliases for output

15. Upload Results (upload_results)
Before: No apparent changes in parameter names, but now uses the argument merging logic.

General summary of changes:

  • Mass renaming of output-related arguments to output_dir (with multiple CLI aliases for backward compatibility).
  • Style change for some arguments (e.g., from kebab-case to snake_case).
  • Centralization of argument merging using the merge_with_extra_config function, standardizing how all commands receive parameters.

⚠️ Action required:

A full round of testing is required for all modules, as some parameter names have changed and strict consistency is now enforced.
When updating your configuration files, always use the internal parameter names (e.g., output_dir, metadata_file, etc.) as shown in initial_config.yaml.

Closes #523

Daniel-VM added a commit to Daniel-VM/relecov-tools that referenced this pull request Jun 3, 2025
Daniel-VM added 28 commits June 6, 2025 13:45
@Daniel-VM
Daniel-VM marked this pull request as ready for review June 9, 2025 13:59
Comment thread README.md Outdated
Comment thread README.md Outdated
Comment thread README.md Outdated
Comment thread README.md Outdated
Comment thread README.md Outdated
Comment thread README.md Outdated
Comment thread README.md Outdated
Comment thread README.md Outdated
Comment thread README.md Outdated
@Daniel-VM
Daniel-VM merged commit 238a540 into BU-ISCIII:develop Jun 10, 2025
17 checks passed
Daniel-VM added a commit that referenced this pull request Jun 10, 2025
Daniel-VM added a commit that referenced this pull request Jun 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants