Skip to content

Update nf-schema examples #141

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: 2.5.0dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/nextflow_schema/nextflow_schema_examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,5 @@ It is used as a test fixture in the nf-schema package [here](https://github.com/
```json
--8<-- "src/testResources/nextflow_schema.json"
```

Even more examples can be found in the plugin [`examples` directory](../../examples) in the GitHub repository.
11 changes: 2 additions & 9 deletions docs/nextflow_schema/sample_sheet_schema_examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,6 @@ description: Example JSON Schema for a Nextflow pipeline `nextflow_schema.json`
The nf-core/rnaseq pipeline was one of the first to have a sample sheet schema.
You can see this, used for validating sample sheets with `--input` here: [`assets/schema_input.json`](https://github.com/nf-core/rnaseq/blob/5671b65af97fe78a2f9b4d05d850304918b1b86e/assets/schema_input.json).

!!! tip

Note the approach used for validating filenames in the `fastq_2` column.
The column is optional, so if a `pattern` was supplied by itself then validation would fail
when no string is supplied.

Instead, we say that the string must _either_ match that pattern or it must have a
`maxLength` of `0` (an empty string).

```json
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
Expand Down Expand Up @@ -71,3 +62,5 @@ It is used as a test fixture in the nf-schema package [here](https://github.com/
```json
--8<-- "src/testResources/schema_input.json"
```

Even more examples can be found in the plugin [`examples` directory](../../examples) in the GitHub repository.
13 changes: 13 additions & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# nf-schema examples

This directory contains example files for the nf-schema project.

1. [Help Message](./helpMessage/) contains a pipeline that generates a help message for a given schema.
2. [Parameter summary log](./paramSummaryLog/) contains a pipeline that generates a summary log of the parameters used in a given schema.
3. [Parameter summary map](./paramSummaryMap/) contains a pipeline that generates a summary map of the parameters used in a given schema. The difference between with the previous example is that this one produces a machine readable data structure instead of a human readable output string.
4. [Basic samplesheet conversion](./samplesheetToListBasic/) contains a pipeline that converts a samplesheet to a list that can be easily converted to a channel for usage in the pipeline.
5. [Samplesheet conversion with meta](./samplesheetToListMeta/) contains a pipeline that converts a samplesheet to a list that can be easily converted to a channel for usage in the pipeline. This example contains a meta map.
6. [Samplesheet conversion order](./samplesheetToListOrder/) contains a pipeline that converts a samplesheet to a list. This example is used to explain the order of the fields in the samplesheet can be changed freely while maintaining the same output list order. The output order is based on the order of the fields in the schema.
7. [Validate Parameters](./validateParameters/) contains a pipeline that validates the parameters used in a given schema.
8. [Fail unrecognized parameters](./failUnrecognizedParameters/) contains a pipeline that fails when unrecognized parameters are used in a given schema.
9. [Warning unrecognized parameters](./warningUnrecognizedParameters/) contains a pipeline that warns when unrecognized parameters are used in a given schema.
7 changes: 7 additions & 0 deletions examples/helpMessage/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Help Message

This example demonstrates how to generate a help message for a given schema using the nf-schema plugin.

There are two example log files provided in this directory:
- `log.txt`: This log file shows the output when running the pipeline with the `--help` option. It contains the help message generated for the schema.
- `log_outdir.txt`: This log file shows the output when running the pipeline with the `--help outdir` option. It contains the help message for the `outdir` parameter specifically.
25 changes: 25 additions & 0 deletions examples/helpMessage/log.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@

N E X T F L O W ~ version 25.04.0

Launching `pipeline/main.nf` [confident_lamarr] DSL2 - revision: 629cc11c05


This is a help message for the pipeline.
It provides information about the parameters and usage.

Typical pipeline command:

nextflow run my_pipeline --input input_file.csv

--help [boolean, string] Show the help message for all top level parameters. When a parameter is given to `--help`, the full help message of that parameter will be printed.
--helpFull [boolean] Show the help message for all non-hidden parameters.
--showHidden [boolean] Show all hidden parameters in the help message. This needs to be used in combination with `--help` or `--helpFull`.

Input/output options
--input [string] Path to comma-separated file containing information about the samples in the experiment.
--outdir [string] The output directory where the results will be saved. You have to use absolute paths to storage on Cloud infrastructure.

------------------------------------------------------

For more information, please refer to the documentation.

Original file line number Diff line number Diff line change
@@ -1,7 +1,13 @@
N E X T F L O W ~ version 23.04.1
Launching `pipeline/main.nf` [exotic_rutherford] DSL2 - revision: 8bf4c8d053

Typical pipeline command:
N E X T F L O W ~ version 25.04.0

Launching `pipeline/main.nf` [agitated_wegener] DSL2 - revision: 629cc11c05


This is a help message for the pipeline.
It provides information about the parameters and usage.

Typical pipeline command:

nextflow run my_pipeline --input input_file.csv

Expand All @@ -10,3 +16,6 @@ Typical pipeline command:
format : directory-path
description: The output directory where the results will be saved. You have to use absolute paths to storage on Cloud infrastructure.
------------------------------------------------------

For more information, please refer to the documentation.

3 changes: 3 additions & 0 deletions examples/helpMessage/pipeline/main.nf
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
workflow {

}
23 changes: 23 additions & 0 deletions examples/helpMessage/pipeline/nextflow.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
plugins {
id '[email protected]'
}

params {
input = "samplesheet.csv"
outdir = "results"
}

validation {
help {
command = "nextflow run my_pipeline --input input_file.csv"
beforeText = """
This is a help message for the pipeline.
It provides information about the parameters and usage.

"""
afterText = """
For more information, please refer to the documentation.
"""
enabled = true
}
}
12 changes: 0 additions & 12 deletions examples/paramsHelp/log.txt

This file was deleted.

6 changes: 0 additions & 6 deletions examples/paramsHelp/pipeline/main.nf

This file was deleted.

8 changes: 0 additions & 8 deletions examples/paramsHelp/pipeline/nextflow.config

This file was deleted.

3 changes: 0 additions & 3 deletions examples/paramsHelp/samplesheet.csv

This file was deleted.

6 changes: 6 additions & 0 deletions examples/paramsSummaryLog/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Parameters summary log

This example demonstrates how to generate a summary log of the parameters used in a given schema using the nf-schema plugin.

There's one example log file provided in this directory:
- `log.txt`: This log file shows the output when running the pipeline.
28 changes: 17 additions & 11 deletions examples/paramsSummaryLog/log.txt
Original file line number Diff line number Diff line change
@@ -1,18 +1,24 @@
N E X T F L O W ~ version 23.04.1
Launching `pipeline/main.nf` [sleepy_goldberg] DSL2 - revision: 7a280216f3

Core Nextflow options
runName : sleepy_goldberg
launchDir : /Users/demo/GitHub/nextflow-io/nf-schema/examples/paramsSummaryLog
workDir : /Users/demo/GitHub/nextflow-io/nf-schema/examples/paramsSummaryLog/work
projectDir : /Users/demo/GitHub/nextflow-io/nf-schema/examples/paramsSummaryLog/pipeline
userName : demo
profile : standard
configFiles:
N E X T F L O W ~ version 25.04.0

Launching `pipeline/main.nf` [angry_poitras] DSL2 - revision: fca518be2f


Input/output options
This is a summary message for the pipeline parameters.
Input/output options
input : samplesheet.csv
outdir : results

Core Nextflow options
runName : angry_poitras
launchDir : /home/nvnieuwk/Documents/nextflow/nf-schema/examples/paramsSummaryLog
workDir : /home/nvnieuwk/Documents/nextflow/nf-schema/examples/paramsSummaryLog/work
projectDir : /home/nvnieuwk/Documents/nextflow/nf-schema/examples/paramsSummaryLog/pipeline
userName : nvnieuwk
profile : standard
configFiles: /home/nvnieuwk/.nextflow/config, /home/nvnieuwk/Documents/nextflow/nf-schema/examples/paramsSummaryLog/pipeline/nextflow.config

!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
For more information, please refer to the documentation.

4 changes: 3 additions & 1 deletion examples/paramsSummaryLog/pipeline/main.nf
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
include { paramsSummaryLog } from 'plugin/nf-schema'

log.info paramsSummaryLog(workflow)
workflow {
log.info paramsSummaryLog(workflow)
}
13 changes: 12 additions & 1 deletion examples/paramsSummaryLog/pipeline/nextflow.config
Original file line number Diff line number Diff line change
@@ -1,8 +1,19 @@
plugins {
id 'nf-schema@2.0.0'
id 'nf-schema@2.5.0'
}

params {
input = "samplesheet.csv"
outdir = "results"
}

validation {
summary {
beforeText = """
This is a summary message for the pipeline parameters.
"""
afterText = """
For more information, please refer to the documentation.
"""
}
}
3 changes: 0 additions & 3 deletions examples/paramsSummaryLog/samplesheet.csv

This file was deleted.

6 changes: 6 additions & 0 deletions examples/paramsSummaryMap/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Parameters summary map

This example demonstrates how to generate a summary map of the parameters used in a given schema using the nf-schema plugin.

There's one example log file provided in this directory:
- `log.txt`: This log file shows the output when running the pipeline, which prints out the parameters summary map.
8 changes: 5 additions & 3 deletions examples/paramsSummaryMap/log.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
N E X T F L O W ~ version 23.04.1
Launching `pipeline/main.nf` [happy_lamport] DSL2 - revision: c45338cd96

[Core Nextflow options:[runName:happy_lamport, launchDir:/Users/ewels/GitHub/nextflow-io/nf-schema/examples/paramsSummaryMap, workDir:/Users/ewels/GitHub/nextflow-io/nf-schema/examples/paramsSummaryMap/work, projectDir:/Users/ewels/GitHub/nextflow-io/nf-schema/examples/paramsSummaryMap/pipeline, userName:ewels, profile:standard, configFiles:], Input/output options:[input:samplesheet.csv, outdir:results]]
N E X T F L O W ~ version 25.04.0

Launching `pipeline/main.nf` [irreverent_stallman] DSL2 - revision: 8bb467530c

[Input/output options:[input:samplesheet.csv, outdir:results], Core Nextflow options:[runName:irreverent_stallman, launchDir:/home/nvnieuwk/Documents/nextflow/nf-schema/examples/paramsSummaryMap, workDir:/home/nvnieuwk/Documents/nextflow/nf-schema/examples/paramsSummaryMap/work, projectDir:/home/nvnieuwk/Documents/nextflow/nf-schema/examples/paramsSummaryMap/pipeline, userName:nvnieuwk, profile:standard, configFiles:/home/nvnieuwk/.nextflow/config, /home/nvnieuwk/Documents/nextflow/nf-schema/examples/paramsSummaryMap/pipeline/nextflow.config]]
4 changes: 3 additions & 1 deletion examples/paramsSummaryMap/pipeline/main.nf
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
include { paramsSummaryMap } from 'plugin/nf-schema'

println paramsSummaryMap(workflow)
workflow {
println paramsSummaryMap(workflow)
}
2 changes: 1 addition & 1 deletion examples/paramsSummaryMap/pipeline/nextflow.config
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
plugins {
id 'nf-schema@2.0.0'
id 'nf-schema@2.5.0'
}

params {
Expand Down
3 changes: 0 additions & 3 deletions examples/paramsSummaryMap/samplesheet.csv

This file was deleted.

6 changes: 6 additions & 0 deletions examples/samplesheetToListBasic/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Basic samplesheet conversion

This example shows how to convert a samplesheet to a list that can be easily converted to a channel for usage in the pipeline.

There's one example log file provided in this directory:
- `log.txt`: This log file shows the output when running the pipeline, which prints out the samplesheet converted to a channel from a list.
2 changes: 0 additions & 2 deletions examples/samplesheetToListBasic/launch.sh

This file was deleted.

7 changes: 5 additions & 2 deletions examples/samplesheetToListBasic/log.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
N E X T F L O W ~ version 23.04.0
Launching `pipeline/main.nf` [distraught_marconi] DSL2 - revision: 74f697a0d9

N E X T F L O W ~ version 25.04.0

Launching `pipeline/main.nf` [elated_almeida] DSL2 - revision: 34f1f7de6e

[mysample1, input1_R1.fq.gz, input1_R2.fq.gz, forward]
[mysample2, input2_R1.fq.gz, input2_R2.fq.gz, forward]
7 changes: 5 additions & 2 deletions examples/samplesheetToListBasic/pipeline/main.nf
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
include { samplesheetToList } from 'plugin/nf-schema'

ch_input = Channel.fromList(samplesheetToList(params.input, "assets/schema_input.json"))
workflow {
ch_input = Channel.fromList(samplesheetToList(params.input, "assets/schema_input.json"))

ch_input.view()
}

ch_input.view()
4 changes: 2 additions & 2 deletions examples/samplesheetToListBasic/pipeline/nextflow.config
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
plugins {
id 'nf-schema@2.0.0'
id 'nf-schema@2.5.0'
}

params {
input = "samplesheet.csv"
input = "${projectDir}/../samplesheet.csv"
output = "results"
}
6 changes: 6 additions & 0 deletions examples/samplesheetToListMeta/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Samplesheet conversion with meta

This example shows how to convert a samplesheet to a list that can be easily converted to a channel for usage in the pipeline. This example contains a meta map.

There's one example log file provided in this directory:
- `log.txt`: This log file shows the output when running the pipeline, which prints out the samplesheet converted to a channel from a list. This contains a meta map.
2 changes: 0 additions & 2 deletions examples/samplesheetToListMeta/launch.sh

This file was deleted.

7 changes: 5 additions & 2 deletions examples/samplesheetToListMeta/log.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
N E X T F L O W ~ version 23.04.0
Launching `pipeline/main.nf` [romantic_kare] DSL2 - revision: 74f697a0d9

N E X T F L O W ~ version 25.04.0

Launching `pipeline/main.nf` [nostalgic_darwin] DSL2 - revision: 2d849659b1

[[my_sample_id:mysample1, my_strandedness:forward], input1_R1.fq.gz, input1_R2.fq.gz]
[[my_sample_id:mysample2, my_strandedness:forward], input2_R1.fq.gz, input2_R2.fq.gz]
6 changes: 4 additions & 2 deletions examples/samplesheetToListMeta/pipeline/main.nf
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
include { samplesheetToList } from 'plugin/nf-schema'

ch_input = Channel.fromList(samplesheetToList(params.input, "assets/schema_input.json"))
workflow {
ch_input = Channel.fromList(samplesheetToList(params.input, "assets/schema_input.json"))

ch_input.view()
ch_input.view()
}
4 changes: 2 additions & 2 deletions examples/samplesheetToListMeta/pipeline/nextflow.config
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
plugins {
id 'nf-schema@2.0.0'
id 'nf-schema@2.5.0'
}

params {
input = "samplesheet.csv"
input = "${projectDir}/../samplesheet.csv"
output = "results"
}
6 changes: 6 additions & 0 deletions examples/samplesheetToListOrder/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Samplesheet conversion order

This example shows how to convert a samplesheet to a list. This example is used to explain the order of the fields in the samplesheet can be changed freely while maintaining the same output list order. The output order is based on the order of the fields in the schema.

There's one example log file provided in this directory:
- `log.txt`: This log file shows the output when running the pipeline.
2 changes: 0 additions & 2 deletions examples/samplesheetToListOrder/launch.sh

This file was deleted.

7 changes: 5 additions & 2 deletions examples/samplesheetToListOrder/log.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
N E X T F L O W ~ version 23.04.0
Launching `pipeline/main.nf` [elated_kowalevski] DSL2 - revision: 74f697a0d9

N E X T F L O W ~ version 25.04.0

Launching `pipeline/main.nf` [magical_wozniak] DSL2 - revision: 2d849659b1

[forward, mysample1, input1_R2.fq.gz, input1_R1.fq.gz]
[forward, mysample2, input2_R2.fq.gz, input2_R1.fq.gz]
6 changes: 4 additions & 2 deletions examples/samplesheetToListOrder/pipeline/main.nf
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
include { samplesheetToList } from 'plugin/nf-schema'

ch_input = Channel.fromList(samplesheetToList(params.input, "assets/schema_input.json"))
workflow {
ch_input = Channel.fromList(samplesheetToList(params.input, "assets/schema_input.json"))

ch_input.view()
ch_input.view()
}
Loading