Skip to content

Nextflow launch does not propagate workspace secrets to Seqera Platform jobs #7039

@rnaidu-seqera

Description

@rnaidu-seqera

Bug report

Expected behavior and actual behavior

When running nextflow launch with a Seqera Platform workspace configured (via ~/.nextflow/config), workspace secrets defined in that Platform workspace should be available to pipeline processes, consistent with the behavior of a LaunchPad-initiated run.

Steps to reproduce the problem

  1. Define one or more workspace secrets in a Seqera Platform workspace (e.g. DRAGEN_USERNAME).
  2. Create a compute environment in that workspace with a Batch Execution Role configured.
  3. Configure ~/.nextflow/config with your Platform access token, endpoint, workspace ID, and compute environment ID:
tower {                                                                                                                          
    accessToken  = '<token>'
    endpoint     = '<endpoint>'                                                                                                  
    workspaceId  = '<workspace_id>'                       
    computeEnvId = '<ce_id>'       
}
  1. Run a pipeline that references a workspace secret:
    nextflow launch <pipeline-repo> -r main -profile awsbatch
  2. Observe that the first process referencing the secret fails.

Program output

Unable to find a secret with name 'DRAGEN_USERNAME' in AWS region: eu-west-2

Environment

  • Nextflow version: [?]
  • Java version: [?]
  • Operating system: [macOS, Linux, etc]
  • Bash version: (use the command $SHELL --version)

Additional context

nextflow launch constructs its API request to /workflow/launch in LaunchCommandImpl.buildLaunchRequestPayload() but never includes userSecrets or workspaceSecrets in the payload. As a result, Platform has no secrets to store in AWS Secrets Manager for the run, yet the Batch job environment always has NXF_ENABLE_SECRETS=true set, causing Nextflow to fail to to fetch secrets that were never stored. This works correctly via the Platform UI LaunchPad because the launch form gathers secret selections from the user and includes them in the same API request. The fix requires adding -user-secret / -workspace-secret CLI flags to CmdLaunch and wiring them into the API payload in LaunchCommandImpl.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions