-
-
Notifications
You must be signed in to change notification settings - Fork 4
Regenerate 2026-01-03 with some CWL extensions #43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mr-c
wants to merge
1
commit into
main
Choose a base branch
from
regen_20260103
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,254 @@ | ||
|
|
||
| import { | ||
| Dictionary, | ||
| expandUrl, | ||
| loadField, | ||
| LoaderInstances, | ||
| LoadingOptions, | ||
| Saveable, | ||
| ValidationException, | ||
| prefixUrl, | ||
| save, | ||
| saveRelativeUri | ||
| } from './util/Internal' | ||
| import { v4 as uuidv4 } from 'uuid' | ||
| import * as Internal from './util/Internal' | ||
|
|
||
|
|
||
| /** | ||
| * Auto-generated class implementation for http://commonwl.org/cwltool#CUDARequirement | ||
| * | ||
| * Require support for NVIDA CUDA (GPU hardware acceleration). | ||
| * | ||
| */ | ||
| export class CUDARequirement extends Saveable implements Internal.CUDARequirementProperties { | ||
| extensionFields?: Internal.Dictionary<any> | ||
|
|
||
| /** | ||
| * cwltool:CUDARequirement | ||
| */ | ||
| class_: string | ||
|
|
||
| /** | ||
| * CUDA hardware capability required to run the software, in X.Y | ||
| * format. | ||
| * | ||
| * * If this is a single value, it defines only the minimum | ||
| * compute capability. GPUs with higher capability are also | ||
| * accepted. | ||
| * | ||
| * * If it is an array value, then only select GPUs with compute | ||
| * capabilities that explicitly appear in the array. | ||
| * | ||
| */ | ||
| cudaComputeCapability: string | Array<string> | ||
|
|
||
| /** | ||
| * Maximum number of GPU devices to request. If not specified, | ||
| * same as `cudaDeviceCountMin`. | ||
| * | ||
| */ | ||
| cudaDeviceCountMax?: undefined | number | string | ||
|
|
||
| /** | ||
| * Minimum number of GPU devices to request. If not specified, | ||
| * same as `cudaDeviceCountMax`. If neither are specified, | ||
| * default 1. | ||
| * | ||
| */ | ||
| cudaDeviceCountMin?: undefined | number | string | ||
|
|
||
| /** | ||
| * Minimum CUDA version to run the software, in X.Y format. This | ||
| * corresponds to a CUDA SDK release. When running directly on | ||
| * the host (not in a container) the host must have a compatible | ||
| * CUDA SDK (matching the exact version, or, starting with CUDA | ||
| * 11.3, matching major version). When run in a container, the | ||
| * container image should provide the CUDA runtime, and the host | ||
| * driver is injected into the container. In this case, because | ||
| * CUDA drivers are backwards compatible, it is possible to | ||
| * use an older SDK with a newer driver across major versions. | ||
| * | ||
| * See https://docs.nvidia.com/deploy/cuda-compatibility/ for | ||
| * details. | ||
| * | ||
| */ | ||
| cudaVersionMin: string | ||
|
|
||
|
|
||
| constructor ({loadingOptions, extensionFields, class_ = 'CUDARequirement', cudaComputeCapability, cudaDeviceCountMax, cudaDeviceCountMin, cudaVersionMin} : {loadingOptions?: LoadingOptions} & Internal.CUDARequirementProperties) { | ||
| super(loadingOptions) | ||
| this.extensionFields = extensionFields ?? {} | ||
| this.class_ = class_ | ||
| this.cudaComputeCapability = cudaComputeCapability | ||
| this.cudaDeviceCountMax = cudaDeviceCountMax | ||
| this.cudaDeviceCountMin = cudaDeviceCountMin | ||
| this.cudaVersionMin = cudaVersionMin | ||
| } | ||
|
|
||
| /** | ||
| * Used to construct instances of {@link CUDARequirement }. | ||
| * | ||
| * @param __doc Document fragment to load this record object from. | ||
| * @param baseuri Base URI to generate child document IDs against. | ||
| * @param loadingOptions Context for loading URIs and populating objects. | ||
| * @param docRoot ID at this position in the document (if available) | ||
| * @returns An instance of {@link CUDARequirement } | ||
| * @throws {@link ValidationException} If the document fragment is not a | ||
| * {@link Dictionary} or validation of fields fails. | ||
| */ | ||
| static override async fromDoc (__doc: any, baseuri: string, loadingOptions: LoadingOptions, | ||
| docRoot?: string): Promise<Saveable> { | ||
| const _doc = Object.assign({}, __doc) | ||
| const __errors: ValidationException[] = [] | ||
|
|
||
| let class_ | ||
| try { | ||
| class_ = await loadField(_doc.class, LoaderInstances.uristrtypeFalseTrueNoneNone, | ||
| baseuri, loadingOptions) | ||
| } catch (e) { | ||
| if (e instanceof ValidationException) { | ||
| __errors.push( | ||
| new ValidationException('the `class` field is not valid because: ', [e]) | ||
| ) | ||
| } else { | ||
| throw e | ||
| } | ||
| } | ||
|
|
||
| let cudaComputeCapability | ||
| try { | ||
| cudaComputeCapability = await loadField(_doc.cudaComputeCapability, LoaderInstances.unionOfstrtypeOrarrayOfstrtype, | ||
| baseuri, loadingOptions) | ||
| } catch (e) { | ||
| if (e instanceof ValidationException) { | ||
| __errors.push( | ||
| new ValidationException('the `cudaComputeCapability` field is not valid because: ', [e]) | ||
| ) | ||
| } else { | ||
| throw e | ||
| } | ||
| } | ||
|
|
||
| let cudaDeviceCountMax | ||
| if ('cudaDeviceCountMax' in _doc) { | ||
| try { | ||
| cudaDeviceCountMax = await loadField(_doc.cudaDeviceCountMax, LoaderInstances.unionOfundefinedtypeOrinttypeOrExpressionLoader, | ||
| baseuri, loadingOptions) | ||
| } catch (e) { | ||
| if (e instanceof ValidationException) { | ||
| __errors.push( | ||
| new ValidationException('the `cudaDeviceCountMax` field is not valid because: ', [e]) | ||
| ) | ||
| } else { | ||
| throw e | ||
| } | ||
| } | ||
| } | ||
|
|
||
| let cudaDeviceCountMin | ||
| if ('cudaDeviceCountMin' in _doc) { | ||
| try { | ||
| cudaDeviceCountMin = await loadField(_doc.cudaDeviceCountMin, LoaderInstances.unionOfundefinedtypeOrinttypeOrExpressionLoader, | ||
| baseuri, loadingOptions) | ||
| } catch (e) { | ||
| if (e instanceof ValidationException) { | ||
| __errors.push( | ||
| new ValidationException('the `cudaDeviceCountMin` field is not valid because: ', [e]) | ||
| ) | ||
| } else { | ||
| throw e | ||
| } | ||
| } | ||
| } | ||
|
|
||
| let cudaVersionMin | ||
| try { | ||
| cudaVersionMin = await loadField(_doc.cudaVersionMin, LoaderInstances.strtype, | ||
| baseuri, loadingOptions) | ||
| } catch (e) { | ||
| if (e instanceof ValidationException) { | ||
| __errors.push( | ||
| new ValidationException('the `cudaVersionMin` field is not valid because: ', [e]) | ||
| ) | ||
| } else { | ||
| throw e | ||
| } | ||
| } | ||
|
|
||
| const extensionFields: Dictionary<any> = {} | ||
| for (const [key, value] of Object.entries(_doc)) { | ||
| if (!CUDARequirement.attr.has(key)) { | ||
| if ((key as string).includes(':')) { | ||
| const ex = expandUrl(key, '', loadingOptions, false, false) | ||
| extensionFields[ex] = value | ||
| } else { | ||
| __errors.push( | ||
| new ValidationException(`invalid field ${key as string}, \ | ||
| expected one of: \`class\`,\`cudaComputeCapability\`,\`cudaDeviceCountMax\`,\`cudaDeviceCountMin\`,\`cudaVersionMin\``) | ||
| ) | ||
| break | ||
| } | ||
| } | ||
| } | ||
|
|
||
| if (__errors.length > 0) { | ||
| throw new ValidationException("Trying 'CUDARequirement'", __errors) | ||
| } | ||
|
|
||
| const schema = new CUDARequirement({ | ||
| extensionFields: extensionFields, | ||
| loadingOptions: loadingOptions, | ||
| class_: class_, | ||
| cudaComputeCapability: cudaComputeCapability, | ||
| cudaDeviceCountMax: cudaDeviceCountMax, | ||
| cudaDeviceCountMin: cudaDeviceCountMin, | ||
| cudaVersionMin: cudaVersionMin | ||
| }) | ||
| return schema | ||
| } | ||
|
|
||
| save (top: boolean = false, baseUrl: string = '', relativeUris: boolean = true) | ||
| : Dictionary<any> { | ||
| const r: Dictionary<any> = {} | ||
| for (const ef in this.extensionFields) { | ||
| r[prefixUrl(ef, this.loadingOptions.vocab)] = this.extensionFields.ef | ||
| } | ||
|
|
||
| if (this.class_ != null) { | ||
| const u = saveRelativeUri(this.class_, baseUrl, false, | ||
| relativeUris, undefined) | ||
| if (u != null) { | ||
| r.class = u | ||
| } | ||
| } | ||
|
|
||
| if (this.cudaComputeCapability != null) { | ||
| r.cudaComputeCapability = save(this.cudaComputeCapability, false, baseUrl, relativeUris) | ||
| } | ||
|
|
||
| if (this.cudaDeviceCountMax != null) { | ||
| r.cudaDeviceCountMax = save(this.cudaDeviceCountMax, false, baseUrl, relativeUris) | ||
| } | ||
|
|
||
| if (this.cudaDeviceCountMin != null) { | ||
| r.cudaDeviceCountMin = save(this.cudaDeviceCountMin, false, baseUrl, relativeUris) | ||
| } | ||
|
|
||
| if (this.cudaVersionMin != null) { | ||
| r.cudaVersionMin = save(this.cudaVersionMin, false, baseUrl, relativeUris) | ||
| } | ||
|
|
||
| if (top) { | ||
| if (this.loadingOptions.namespaces != null) { | ||
| r.$namespaces = this.loadingOptions.namespaces | ||
| } | ||
| if (this.loadingOptions.schemas != null) { | ||
| r.$schemas = this.loadingOptions.schemas | ||
| } | ||
| } | ||
| return r | ||
| } | ||
|
|
||
| static attr: Set<string> = new Set(['class','cudaComputeCapability','cudaDeviceCountMax','cudaDeviceCountMin','cudaVersionMin']) | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
|
|
||
| import * as Internal from './util/Internal' | ||
|
|
||
|
|
||
| /** | ||
| * Auto-generated interface for http://commonwl.org/cwltool#CUDARequirement | ||
| * | ||
| * Require support for NVIDA CUDA (GPU hardware acceleration). | ||
| * | ||
| */ | ||
| export interface CUDARequirementProperties extends Internal.ProcessRequirementProperties { | ||
|
|
||
| extensionFields?: Internal.Dictionary<any> | ||
|
|
||
| /** | ||
| * cwltool:CUDARequirement | ||
| */ | ||
| class_?: string | ||
|
|
||
| /** | ||
| * CUDA hardware capability required to run the software, in X.Y | ||
| * format. | ||
| * | ||
| * * If this is a single value, it defines only the minimum | ||
| * compute capability. GPUs with higher capability are also | ||
| * accepted. | ||
| * | ||
| * * If it is an array value, then only select GPUs with compute | ||
| * capabilities that explicitly appear in the array. | ||
| * | ||
| */ | ||
| cudaComputeCapability: string | Array<string> | ||
|
|
||
| /** | ||
| * Maximum number of GPU devices to request. If not specified, | ||
| * same as `cudaDeviceCountMin`. | ||
| * | ||
| */ | ||
| cudaDeviceCountMax?: undefined | number | string | ||
|
|
||
| /** | ||
| * Minimum number of GPU devices to request. If not specified, | ||
| * same as `cudaDeviceCountMax`. If neither are specified, | ||
| * default 1. | ||
| * | ||
| */ | ||
| cudaDeviceCountMin?: undefined | number | string | ||
|
|
||
| /** | ||
| * Minimum CUDA version to run the software, in X.Y format. This | ||
| * corresponds to a CUDA SDK release. When running directly on | ||
| * the host (not in a container) the host must have a compatible | ||
| * CUDA SDK (matching the exact version, or, starting with CUDA | ||
| * 11.3, matching major version). When run in a container, the | ||
| * container image should provide the CUDA runtime, and the host | ||
| * driver is injected into the container. In this case, because | ||
| * CUDA drivers are backwards compatible, it is possible to | ||
| * use an older SDK with a newer driver across major versions. | ||
| * | ||
| * See https://docs.nvidia.com/deploy/cuda-compatibility/ for | ||
| * details. | ||
| * | ||
| */ | ||
| cudaVersionMin: string | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,23 +1,4 @@ | ||
|
|
||
| export enum CWLVersion { | ||
| DRAFT_2='draft-2', | ||
| DRAFT_3_DEV1='draft-3.dev1', | ||
| DRAFT_3_DEV2='draft-3.dev2', | ||
| DRAFT_3_DEV3='draft-3.dev3', | ||
| DRAFT_3_DEV4='draft-3.dev4', | ||
| DRAFT_3_DEV5='draft-3.dev5', | ||
| DRAFT_3='draft-3', | ||
| DRAFT_4_DEV1='draft-4.dev1', | ||
| DRAFT_4_DEV2='draft-4.dev2', | ||
| DRAFT_4_DEV3='draft-4.dev3', | ||
| V1_0_DEV4='v1.0.dev4', | ||
| V1_0='v1.0', | ||
| V1_1_0_DEV1='v1.1.0-dev1', | ||
| V1_1='v1.1', | ||
| V1_2_0_DEV1='v1.2.0-dev1', | ||
| V1_2_0_DEV2='v1.2.0-dev2', | ||
| V1_2_0_DEV3='v1.2.0-dev3', | ||
| V1_2_0_DEV4='v1.2.0-dev4', | ||
| V1_2_0_DEV5='v1.2.0-dev5', | ||
| V1_2='v1.2', | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would adding this back in resolve this ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried that, but then there are other validation issues: https://github.com/common-workflow-lab/cwl-ts-auto/actions/runs/20676696133/job/59365273384
The removal is legitimate, as the class is now abstract..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes but the object constructor still expects the extension fields, such as https://github.com/common-workflow-lab/cwl-ts-auto/blob/regen_20260103/src/CommandLineBindable.ts#L88
So you either need to either remove the extension fields from those types of calls, OR update the constructor type to add in the extension fields type (which was previously just in the CommandLineBindable properties), like so
This constructor
to
Yep because id is now set to 'mandatory' in many of the fields, such as https://github.com/common-workflow-lab/cwl-ts-auto/pull/43/changes#diff-07abac72669befd367f8ae59ac50db4ca16d87201905fb78ab737201b14488a6R27 but none of the cwl tests have 'id' in their attributes? If this is intentional, we will need to update the script that generates the cwl schema to ensure that the 'id' attribute is removed from the required fields, which shouldn't be too difficult.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds like some of the SALAD processing for maps isn't happening, either on the Typescript side or the JSON schema generation/testing side.
https://www.commonwl.org/v1.2/Workflow.html#map
For example, the
idfield forWorkflowStepInput(used by theinfield of a workflow step) is rarely in the long form:but usually written in the short form:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can manually remove the 'id' fields in these sections from the json schema, that's why we generate the cwl-raw file and then the curated schema from that as the typescript isn't a perfect representation of a CWL file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The JSON schema should be modified to support all variations of the map syntax. I guess that information isn't present in the TypeScript objects because the loader functions take care of the variations. So the post-processing from the TypeScript to JSON Schema is going to need additional information on where to encode the map varients.
That information is in the SALAD definition of CWL.
I see at least 3 options to move forward:
A. Record the information about the map field variations during the TypeScript code generation that produced this repository. Use that information in a post-processing step after the TypeScript objects to JSON Schema generation.
or
B. Create
schema-salad-tool --codegen json-schemato directly output JSON schema, taking into account all themapfield varients.or
C. Create manual patches to apply to the generated JSON Schema, updating them as the CWL 1.2 schema continues to be refined, and manually porting them for the future CWL versions.
I feel like Option B is the most future-proofed and brings value to other SALAD users, but also the most work. In some ways it won't be as difficult as the other additions of new codegen targets as we have two examples of what JSON Schema for CWL would look like.
I can assist with all of these options, but I'm not available to do any of these options entirely myself.