Skip to content

feat: Agat prepare genome resources meta-wrapper#5437

Open
tdayris wants to merge 4 commits into
snakemake:masterfrom
tdayris:agat_prepare_genome_resources
Open

feat: Agat prepare genome resources meta-wrapper#5437
tdayris wants to merge 4 commits into
snakemake:masterfrom
tdayris:agat_prepare_genome_resources

Conversation

@tdayris

@tdayris tdayris commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

This PR is here as an attempt to answer a coworker's issue with pathvars. pathvar evaluations are postponed after the wrapper evaluation. This lead to two kind of errors :

  1. A wrapper expects numbers and pathvars can only be strings. e.g. "v5.10.0/bio/reference/ensembl-sequence" expects numbers in its release parameter.
WorkflowError in file "/tmp/tmpvvs9wown/tmpk101_r_n/test/Snakefile", line 6:
Pathvars have to be a mapping of str to str, with keys being valid pathvar names (i.e. alphanumeric + _, lower-case, no leading number). The following entries are invalid: release='105'
  1. A wrapper can try to process pathvars before they're being replaced by Snakemake. e.g. "v5.10.0/bio/reference/ensembl-sequence" fails to cast release to int.
[Mon Jul  6 12:53:43 2026]
localrule agat_genome_statistics_download_ensembl_genome_annotations:
    output: resources/saccharomyces_cerevisiae.R64-1-1.105/R64-1-1.ensembl.gff3
    log: logs/download_ensembl_genome_annotations/saccharomyces_cerevisiae.R64-1-1.105.log
    jobid: 4
    reason: Missing output files: resources/saccharomyces_cerevisiae.R64-1-1.105/R64-1-1.ensembl.gff3
    resources: tmpdir=/tmp
Shell command: None
Activating conda environment: .snakemake/conda/7d2d8499f7bbaec1feb944bfa4ac99b8_
Traceback (most recent call last):
  File "/tmp/tmp82s2l863/tmpn55elvoc/test/.snakemake/scripts/tmps3eznouy.wrapper.py", line 19, in <module>
    release = int(snakemake.params.release)
ValueError: invalid literal for int() with base 10: '<release>'

Here, we use pathvars and wildcards alongside. But replacing the wildcards with pathvars will reproduce the above issues.

QC

While the contributions guidelines are more extensive, please particularly ensure that:

  • test.py was updated to call any added or updated example rules in a Snakefile
  • input: and output: file paths in the rules can be chosen arbitrarily
  • wherever possible, command line arguments are inferred and set automatically (e.g. based on file extensions in input: or output:)
  • temporary files are either written to a unique hidden folder in the working directory, or (better) stored where the Python function tempfile.gettempdir() points to
  • the meta.yaml contains a link to the documentation of the respective tool or command under url:
  • conda environments use a minimal amount of channels and packages, in recommended ordering

Summary by CodeRabbit

  • New Features
    • Added a new genome statistics workflow that downloads genome data, filters contigs, applies annotation fixes, and produces summary reports with plots.
    • Introduced a test configuration for running the workflow on selected contigs.
    • Added automated coverage for the new workflow to verify expected outputs are generated.

@coderabbitai

coderabbitai Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

This PR adds a new meta-wrapper agat_genome_statistics under meta/bio/, including metadata, a Snakemake workflow chaining Ensembl genome/annotation download, contig filtering, AGAT-based GFF fixing and TSL filtering, contig consistency checks, and genome-wide statistics computation, plus test configuration and a pytest test case.

Changes

Agat genome statistics meta-wrapper

Layer / File(s) Summary
Workflow metadata
meta/bio/agat_genome_statistics/meta.yaml
Defines documentation URL, description table, authors, and default pathvars for resources/results/logs.
Genome download and contig filtering
meta/bio/agat_genome_statistics/meta_wrapper.smk
Adds rules to download Ensembl genome FASTA and GFF3, and optionally filter FASTA to configured contigs.
GFF fixing and TSL filtering
meta/bio/agat_genome_statistics/meta_wrapper.smk
Adds rules to fix known Ensembl GFF format issues via AGAT and remove features with transcript_support_level = "NA".
Contig consistency and statistics
meta/bio/agat_genome_statistics/meta_wrapper.smk
Ensures GFF and FASTA contigs match, then computes genome-wide statistics via agat_sp_statistics.pl, producing report, YAML, and plots.
Test wiring and wrapper listing
meta/bio/agat_genome_statistics/test/Snakefile, meta/bio/agat_genome_statistics/test/config.yaml, meta/bio/agat_genome_statistics/used_wrappers.yaml, test_wrappers.py
Adds test Snakefile aliasing module rules, test config with contigs, used wrappers list, and a pytest test invoking the meta-wrapper.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Snakemake
  participant Download as Download rules
  participant AGAT as AGAT fix/filter rules
  participant Stats as compute_genome_wide_statistics

  Snakemake->>Download: download_ensembl_genome_sequence / download_ensembl_genome_annotations
  Download->>Download: filter_genome_sequence_contigs (optional)
  Download->>AGAT: fix_known_format_issues_with_ensembl_gff
  AGAT->>AGAT: remove_tsl_na_from_gff_annotations
  AGAT->>AGAT: ensure_annotations_and_sequences_have_same_contigs
  AGAT->>Stats: matched GFF + FASTA
  Stats-->>Snakemake: report.txt, statistics.yaml, plots/
Loading

Suggested reviewers: johanneskoester

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title is concise, conventional, and matches the main change: adding an AGAT genome resources meta-wrapper.
Description check ✅ Passed The description covers the PR purpose and includes the required QC checklist with all items checked.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@meta/bio/agat_genome_statistics/meta_wrapper.smk`:
- Around line 59-70: The AGAT filtering command in
remove_tsl_na_from_gff_annotations is passing a quoted value for
transcript_support_level, so it will not match plain NA entries in the GFF3
annotations. Update the params.extra argument for
agat_sp_filter_feature_by_attribute_value.pl to use the unquoted NA value,
keeping the existing attribute and test options intact so the rule matches the
intended records.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 14151a4e-3645-43a5-b0aa-e1bdca3a6e0b

📥 Commits

Reviewing files that changed from the base of the PR and between 75aa7c0 and e05e975.

📒 Files selected for processing (6)
  • meta/bio/agat_genome_statistics/meta.yaml
  • meta/bio/agat_genome_statistics/meta_wrapper.smk
  • meta/bio/agat_genome_statistics/test/Snakefile
  • meta/bio/agat_genome_statistics/test/config.yaml
  • meta/bio/agat_genome_statistics/used_wrappers.yaml
  • test_wrappers.py

Comment thread meta/bio/agat_genome_statistics/meta_wrapper.smk
@fgvieira

fgvieira commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Not sure I follow... this PR add a meta-wrapper for AGAT showcasing how to use wildcards and pathvars together.
About the issues you point out above, are they snakemake bugs? If so, maybe we should report them in the repo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants