-
-
Notifications
You must be signed in to change notification settings - Fork 24
Description
requests to fix any of the below should go against https://github.com/common-workflow-language/cwl-v1.2/tree/1.2.1_proposed You can preview all the implemented changes for v1.2.1 so far but know that they have not been ratified by the CWL Leadership Team; additional comments are requested.
Anyone is welcome to take one of these and start adding the needed conformance tests and specification clarifications! Please just drop a note in the linked issue when you start.
If you have never written a CWL conformance test or edited the text of the CWL standards before, that's okay! Just ask for instructions on what to do in the linked issue of your choice.
Candidates issues
Rough consensus with a known solution
New conformance tests
See https://github.com/common-workflow-language/cwl-v1.2/blob/main/CONFORMANCE_TESTS.md#format-of-the-conformance-test-file for information on how to write a new conformance test
ToDo
- Conformance tests for renaming output files with basename
- confirm that fixing the above also covers everything in Clarify the description of
basename
modification in the spec
- confirm that fixing the above also covers everything in Clarify the description of
- Add tests for itemSeparator
- Conformance test for optional record input
- need conformance test for workflow output + linkMerge: merge_flattened
- There are no specific conformance tests about the behavior for nested inputBinding
- Add conformance test for ECMAScript 5.1 strict mode
- create conformance test for InitialWorkDirRequirement w/ & w/o Docker and reference to input file paths
- Continue with Audit the objects available for every place we allow parameter references/expressions producing new conformance tests for each category
- new conformance test: symlinks using absolute paths in a Directory
- two directory inputs with same basename but different paths
-
dockerLoad
,dockerFile
, anddockerImport
lack conformance tests - workflow in subdirectory, tools in another subdirectory,
run
contains relative paths likerun: ../tools/my_tool.cwl
. Example; source: Run CWL conformance tests via WES DataBiosphere/toil#4052 (comment) - test for parameter references in CommandLineTool.arguments (pull request)
a proper test for WorkReuse(not possible with cwltest)- potential secondaryFiles conformance test candidate (cannot reproduce with cwltool)
Specification clarification
changes to wording of the specification, may require new conformance test(s) to clarify or confirm
ToDo
- Specification is unclear on whether array typeDefs are allowed in SchemaDefRequirement
- Document that Directory loadListing behavior in v1.1 is not backwards compatible
- Clarify records with multiple inputBindings
- Several terms are used for the value of record type: objects, maps, and records
- Define decoding method for loadContents and cwl.output.json
- clarify use of parameter references / expressions in ResourceRequirement fields
- Does CommandLineBinding.loadContents apply to secondary files?
- Expected behavior of outputBinding on array of file
- be explicit about how to format an input object file via the optional cwl-runner CLI
- Improve explanations about array level and per-item inputBinding
- better document the 'default' and 'valueFrom' fields on WorkflowStepInput
- clarify type checking rules
- WorkflowOutputParameter outputSource should not be optional
- Examples for linkMerge merge_flattened
- specify YAML version, which schema
- fields not specified in an input object get a value of 'null'
- clarify sort order for
glob
s - clarify LoadListingEnum
- [prose] Confirm that final output objects must have the listings of all directories recursively expanded
- explain how to render a
type: enum
on the command line - clarify evaluation policy of Workflow level requirements
- Explain the concept of a Abstract CWL Workflow, a
class: Workflow
with one or moreclass: Operation
steps (or sub-steps). An Abstract CWL Workflow description is not (fully) executable, asOperation
processes are themselves abstract; but it does allow describing a workflow and providing a skeleton to attach metadata about the steps, authorship, tools used, and more. Abstract CWL Workflows are used in the RO-Crate workflows and workflow runs profiles; the former used by workflowhub.eu.
Bonus issues
These are worthy, but we don't have a solution ready or obvious plan yet
Pending Discussion:
ToDo
- JS expression returning Undefined
- document format schema bridging technique
- Definition of
hints
is inconsistent - Clarify, make explicit, and add conformance test if CommandLineTool requires a baseCommand vs being a no-op
- default values with secondaryfiles defined by extension needs a conformance test
- "Edit This Section" button for the rendered CWL standards
Done
- outputEval: "time passed" syntax error
- Allow
[]
?
in types under ArraySchema.list - conformance tests: add
docker.io/
todockerPull
entries that lack an explicit registry - clarify $(..) vs ${...}
- conformance test requires
location: Any
what does it mean ? (in progress at Document writing a new conformance test #124) - testing InitialWorkDirRequirement with directories as output and more (in progress at test for nested directories and initial workdir requirement #126 )
- Conflict between the conformance test 61 and the spec for parameter references (in progress at Clarify parameter reference: null and array length #64)
- new conformance test: $(runtime.outdir) for outputBinding glob Add conformance test that verifies we can use runtime outdir with output glob #164
- Unclear specification of
run
field in the workflow description - mention
cwl-upgrader
; explain version mixing Add note about cwl-upgrader in the Workflow changelog #172 - fix the broken InitialWorkDirRequirement link
- remove the
initial_workdir_output_glob_duplicate
test Typo fixes common-workflow-language#160 - The ID numbers are off by one at 220 or before Irrelvant due to Move label to id cwltest#110
- conformance test for input record + JS Add a conformance test for InlineJavascriptRequirement with an input record #170
- New SchemaDefRequirement conformance test candidate Add test for SchemaDefRequirement with a workflow and the types containing an $import statement #171
- conformance test needed: MultipleInputFeatureRequirement on workflow outputs Add a conformance test for MultipleInputFeatureRequirement on Workflow outputs #169
- Define the concept of an "opaque string" Add glossary and clarification on opaque strings and identifiers #162
- affirm that WorkflowOutputParameter.outputSource can reference workflow level inputs Mention that we can refer to workflow level inputs in output parameter outputSource #178
- Stdout behavior with chained commands Add new test for chained commands, update the specification wording #190
- clarify meaning of Process.id Clarify process id field ($graph only) #166
- Why conformance test 63 expect empty result ? Update the doc and use "true" as base command for the metadata test #184
- complicated IWDRs & symlinks #193
- need conformance test for multiple entries in a WorkflowOutputParameter.outputSource #239
- Prose for
location
in File/Directory mentions URIs instead of IRIs - About specification when output class is File
- reparent file specific fields from SchemaBase to Parameter common-workflow-language#232