diff --git a/docs/nextflow_run/00_orientation.md b/docs/nextflow_run/00_orientation.md
new file mode 100644
index 000000000..124840a73
--- /dev/null
+++ b/docs/nextflow_run/00_orientation.md
@@ -0,0 +1,80 @@
+# Orientation
+
+This orientation assumes you have already opened the training environment by clicking on the "Open in GitHub Codespaces" button.
+If not, please do so now, ideally in a second browser window or tab so you can refer back to these instructions.
+
+[](https://codespaces.new/nextflow-io/training?quickstart=1&ref=master)
+
+## GitHub Codespaces
+
+The GitHub Codespaces environment contains all the software, code and data necessary to work through this training course, so you don't need to install anything yourself.
+However, you do need a (free) GitHub account to log in, and you should take a few minutes to familiarize yourself with the interface.
+
+If you have not yet done so, please go through the [Environment Setup](../../envsetup/) mini-course before going any further.
+
+## Working directory
+
+Throughout this training course, we'll be working in the `nextflow-run/` directory.
+
+Change directory now by running this command in the terminal:
+
+```bash
+cd nextflow-run/
+```
+
+!!!tip
+
+ If for whatever reason you move out of this directory, you can always use the full path to return to it, assuming you're running this within the GitHub Codespaces training environment:
+
+ ```bash
+ cd /workspaces/training/nextflow-run
+ ```
+
+Now let's have a look at the contents of this directory.
+
+## Materials provided
+
+You can explore the contents of this directory by using the file explorer on the left-hand side of the training workspace.
+Alternatively, you can use the `tree` command.
+
+Throughout the course, we use the output of `tree` to represent directory structure and contents in a readable form, sometimes with minor modifications for clarity.
+
+Here we generate a table of contents to the second level down:
+
+```bash
+tree . -L 2
+```
+
+If you run this inside `nextflow-run`, you should see the following output:
+
+```console title="Directory contents"
+.
+├── 1-hello.nf
+├── 2a-inputs.nf
+├── 2b-multistep.nf
+├── 2c-modules.nf
+├── 2d-container.nf
+├── 3-main.nf
+├── modules
+│ ├── collectGreetings.nf
+│ ├── convertToUpper.nf
+│ ├── cowpy.nf
+│ └── sayHello.nf
+├── nextflow.config
+└── test-params.yaml
+
+1 directory, 12 files
+```
+
+**Here's a summary of what you should know to get started:**
+
+- **The `.nf` files** are workflow scripts that are numbered based on what part of the course they're used in.
+
+- **The file `nextflow.config`** is a configuration file that sets minimal environment properties.
+ You can ignore it for now.
+
+- **The file `greetings.csv`** contains input data we'll use in most of the course. It is described in Part 2, when we introduce it for the first time.
+
+- **The file `test-params.yaml`** is a file we'll use in Part 3. You can ignore it for now.
+
+**Now, to begin the course, click on the arrow in the bottom right corner of this page.**
diff --git a/docs/nextflow_run/01_basics.md b/docs/nextflow_run/01_basics.md
new file mode 100644
index 000000000..53891e170
--- /dev/null
+++ b/docs/nextflow_run/01_basics.md
@@ -0,0 +1,557 @@
+# Part 1: Run basic operations
+
+In this first part of the Nextflow Run training course, we ease into the topic with a very basic domain-agnostic Hello World example, which we'll use to demonstrate essential operations and point out the corresponding Nextflow code components.
+
+!!! Tip
+
+ A "Hello World!" is a minimalist example that is meant to demonstrate the basic syntax and structure of a programming language or software framework.
+ The example typically consists of printing the phrase "Hello, World!" to the output device, such as the console or terminal, or writing it to a file.
+
+## 0. Warmup: Run Hello World directly
+
+Let's demonstrate this with a simple command that we run directly in the terminal, to show what it does before we wrap it in Nextflow.
+
+!!! Tip
+
+ Remember that you should now be inside the `nextflow-run/` directory as described in the Orientation.
+
+### 0.1. Make the terminal say hello
+
+```bash
+echo 'Hello World!'
+```
+
+This outputs the text 'Hello World' to the terminal.
+
+```console title="Output"
+Hello World!
+```
+
+### 0.2. Now make it write the text output to a file
+
+```bash
+echo 'Hello World!' > output.txt
+```
+
+This does not output anything to the terminal.
+
+```console title="Output"
+
+```
+
+### 0.3. Show the file contents
+
+```bash
+cat output.txt
+```
+
+The text 'Hello World' is now in the output file we specified.
+
+```console title="output.txt" linenums="1"
+Hello World!
+```
+
+!!! Tip
+
+ In the training environment, you can also find the output file in the file explorer, and view its contents by clicking on it.
+
+### Takeaway
+
+You now know how to run a simple command in the terminal that outputs some text, and optionally, how to make it write the output to a file.
+
+### What's next?
+
+Find out what it takes to run a Nextflow workflow that achieves the same result.
+
+---
+
+## 1. Run the workflow
+
+We provide you with a workflow script named `1-hello.nf` that takes an input via a command-line argument named `--greeting` and produces a text file containing that greeting.
+We're not going to look at the code yet; first let's see what it looks like to run it.
+
+### 1.1. Launch the workflow and monitor execution
+
+In the terminal, run the following command:
+
+```bash
+nextflow run 1-hello.nf --greeting 'Hello World!'
+```
+
+You console output should look something like this:
+
+```console title="Output" linenums="1"
+ N E X T F L O W ~ version 25.04.3
+
+Launching `1-hello.nf` [goofy_torvalds] DSL2 - revision: c33d41f479
+
+executor > local (1)
+[a3/7be2fa] sayHello | 1 of 1 ✔
+```
+
+Congratulations, you just ran your first Nextflow workflow!
+
+The most important output here is the last line (line 6):
+
+```console title="Output" linenums="6"
+[a3/7be2fa] sayHello | 1 of 1 ✔
+```
+
+This tells us that the `sayHello` process was successfully executed once (`1 of 1 ✔`).
+
+That's great, but you may be wondering: where is the output?
+
+### 1.2. Find the output file in the `results` directory
+
+This workflow is configured to publish its output to a directory called `results`.
+If you look at your current directory, you will see that when you ran the workflow, Nextflow created a new directory called `results`, which contains a file called `output.txt`.
+
+```console title="results/" linenums="1"
+results
+└── output.txt
+```
+
+Open the file; the contents should match the string you specified on the command line.
+
+
+ File contents
+
+```console title="results/output.txt" linenums="1"
+Hello World!
+```
+
+
+
+That's great, our workflow did what it was supposed to do!
+
+However, be aware that the 'published' result is a copy (or in some cases a symlink) of the actual output produced by Nextflow when it executed the workflow.
+
+So now, we are going to peek under the hood to see where Nextflow actually executed the work.
+
+!!! Warning
+
+ Not all workflows will be set up to publish outputs to a results directory, and/or the directory name may be different.
+ A little further in this section, we will show you how to find out where this behavior is specified.
+
+### 1.3. Find the original output and logs in the `work/` directory
+
+When you run a workflow, Nextflow creates a distinct 'task directory' for every single invocation of each process in the workflow (=every step in the pipeline).
+For each one, it will stage the necessary inputs, execute the relevant instruction(s) and write outputs and log files within that one directory, which is named automatically using a hash in order to make it unique.
+
+All of these task directories will live under a directory called `work` within your current directory (where you're running the command).
+
+That may sound confusing, so let's see what that looks like in practice.
+
+Going back to the console output for the workflow we ran earlier, we had this line:
+
+```console title="Excerpt of command output" linenums="6"
+[a3/7be2fa] sayHello | 1 of 1 ✔
+```
+
+See how the line starts with `[a3/7be2fa]`?
+That is a truncated form of the task directory path for that one process call, and tells you where to find the output of the `sayHello` process call within the `work/` directory path.
+
+You can find the full path by typing the following command (replacing `a3/7be2fa` with what you see in your own terminal) and pressing the tab key to autocomplete the path or adding an asterisk:
+
+```bash
+ls work/a3/7be2fa*
+```
+
+This should yield the full path directory path: `work/a3/7be2fa7be2fad5e71e5f49998f795677fd68`
+
+Let's take a look at what's in there.
+
+!!! Tip
+
+ If you browse the contents of the task subdirectory in the VSCode file explorer, you'll see all the files right away.
+ However, the log files are set to be invisible in the terminal, so if you want to use `ls` or `tree` to view them, you'll need to set the relevant option for displaying invisible files.
+
+ ```bash
+ tree -a work
+ ```
+
+The exact subdirectory names will be different on your system.
+
+
+ Directory contents
+
+```console title="work/"
+work
+└── a3
+ └── 7be2fad5e71e5f49998f795677fd68
+ ├── .command.begin
+ ├── .command.err
+ ├── .command.log
+ ├── .command.out
+ ├── .command.run
+ ├── .command.sh
+ ├── .exitcode
+ └── output.txt
+```
+
+
+
+You should immediately recognize the `output.txt` file, which is in fact the original output of the `sayHello` process that got published to the `results` directory.
+If you open it, you will find the `Hello World!` greeting again.
+
+
+ File contents
+
+```console title="work/a3/7be2fa7be2fad5e71e5f49998f795677fd68/output.txt" linenums="1"
+Hello World!
+```
+
+
+
+So what about all those other files?
+
+These are the helper and log files that Nextflow wrote as part of the task execution:
+
+- **`.command.begin`**: Sentinel file created as soon as the task is launched.
+- **`.command.err`**: Error messages (`stderr`) emitted by the process call
+- **`.command.log`**: Complete log output emitted by the process call
+- **`.command.out`**: Regular output (`stdout`) by the process call
+- **`.command.run`**: Full script run by Nextflow to execute the process call
+- **`.command.sh`**: The command that was actually run by the process call
+- **`.exitcode`**: The exit code resulting from the command
+
+The `.command.sh` file is especially useful because it shows you the main command Nextflow executed not including all the bookkeeping and task/environment setup.
+
+
+ File contents
+
+```console title="work/a3/7be2fa7be2fad5e71e5f49998f795677fd68/command.sh" linenums="1"
+#!/bin/bash -ue
+echo 'Hello World!' > output.txt
+
+```
+
+
+
+So this confirms that the workflow composed the same command we ran directly on the command-line earlier.
+
+!!! Tip
+
+ When something goes wrong and you need to troubleshoot what happened, it can be useful to look at the `command.sh` script to check exactly what command Nextflow composed based on the workflow instructions, variable interpolation and so on.
+
+### 1.4. Optional exercise: re-run with different greetings
+
+Try re-running the workflow a few times with different values for the `--greeting` argument, then look at both the contents of the `results/` directory and the task directories.
+
+Observe how the outputs and logs of isolated task directories are preserved, whereas the contents of the `results` directory are overwritten by the output of subsequent executions.
+
+### Takeaway
+
+You know how to run a simple Nextflow script, monitor its execution and find its outputs.
+
+### What's next?
+
+Learn how to read a basic Nextflow script and identify how its components relate to its functionality.
+
+---
+
+## 2. Examine the Hello World workflow starter script
+
+What we did there was basically treating the workflow script like a black box.
+Now that we've seen what it does, let's open the box and look inside.
+
+_The goal here is not to memorize the syntax of Nextflow code, but to form some basic intuition of what are the main components and how they are organized._
+
+### 2.1. Examine the overall code structure
+
+Let's open the `1-hello.nf` script in the editor pane.
+
+
+ Code
+
+```groovy title="1-hello.nf" linenums="1"
+#!/usr/bin/env nextflow
+
+/*
+ * Use echo to print a greeting to a file
+ */
+process sayHello {
+
+ publishDir 'results', mode: 'copy'
+
+ input:
+ val greeting
+
+ output:
+ path 'output.txt'
+
+ script:
+ """
+ echo '$greeting' > output.txt
+ """
+}
+
+workflow {
+
+ // emit a greeting
+ sayHello(params.greeting)
+}
+```
+
+
+
+A Nextflow script involves two main types of core components: one or more **processes**, and the **workflow** itself.
+Each **process** describes what operation(s) the corresponding step in the pipeline should accomplish, while the **workflow** describes the dataflow logic that connects the various steps.
+
+Let's take a closer look at the **process** block first, then we'll look at the **workflow** block.
+
+### 2.2. The `process` definition
+
+The first block of code describes a **process**.
+The process definition starts with the keyword `process`, followed by the process name and finally the process body delimited by curly braces.
+The process body must contain a script block which specifies the command to run, which can be anything you would be able to run in a command line terminal.
+
+Here we have a **process** called `sayHello` that takes an **input** variable called `greeting` and writes its **output** to a file named `output.txt`.
+
+
+ Code
+
+```groovy title="1-hello.nf" linenums="3"
+/*
+ * Use echo to print a greeting to a file
+ */
+process sayHello {
+
+ publishDir 'results', mode: 'copy'
+
+ input:
+ val greeting
+
+ output:
+ path 'output.txt'
+
+ script:
+ """
+ echo '$greeting' > output.txt
+ """
+}
+```
+
+
+
+This is a very minimal process definition that just contains an `input` definition, an `output` definition and the `script` to execute.
+
+The `input` definition includes the `val` qualifier, which tells Nextflow to expect a value of some kind (can be a string, a number, whatever).
+
+The `output` definition includes the `path` qualifier, which tells Nextflow this should be handled as a path (includes both directory paths and files).
+
+!!! Tip
+
+ The output definition does not _determine_ what output will be created.
+ It simply _declares_ where to find the expected output file(s), so that Nextflow can look for it once execution is complete.
+
+ This is necessary for verifying that the command was executed successfully and for passing the output to downstream processes if needed.
+ Output produced that doesn't match what is declared in the output block will not be passed to downstream processes.
+
+In a real-world pipeline, a process usually contains additional information uch as process directives, which we'll introduce in a little bit.
+
+### 2.3. The `workflow` definition
+
+The second block of code describes the **workflow** itself.
+The workflow definition starts with the keyword `workflow`, followed by an optional name, then the workflow body delimited by curly braces.
+
+Here we have a **workflow** that consists of one call to the `sayHello` process, which takes an input, `params.greeting`, which holds the value we gave to the `--greeting` parameter.
+
+```groovy title="1-hello.nf" linenums="22"
+workflow {
+
+ // emit a greeting
+ sayHello(params.greeting)
+}
+```
+
+This is a very minimal **workflow** definition.
+In a real-world pipeline, the workflow typically contains multiple calls to **processes** connected by **channels**, and there may be default values set up for the variable inputs.
+
+We'll look into that in Part 2 of the course.
+
+### 2.4. The `params` system of command-line parameters
+
+The `params.greeting` we provide to the `sayHello()` process call is a neat bit of Nextflow code and is worth spending an extra minute on.
+
+As mentioned above, that's how we pass the value of the `--greeting` command-line parameter to the `sayHello()` process call.
+In fact, simply declaring `params.someParameterName` will enable us to give the workflow a parameter named `--someParameterName` from the command-line.
+
+!!! Tip
+
+ These workflow parameters declared using the `params` system always take two dashes (`--`).
+ This distinguishes them from Nextflow-level parameters, which only take one dash (`-`).
+
+### Takeaway
+
+You now know how a simple Nextflow workflow is structured, and how the basic components relate to its functionality.
+
+### What's next?
+
+Learn to manage your workflow executions conveniently.
+
+---
+
+## 3. Manage workflow executions
+
+Knowing how to launch workflows and retrieve outputs is great, but you'll quickly find there are a few other aspects of workflow management that will make your life easier.
+
+Here we show you how to take advantage of the `resume` feature for when you need to re-launch the same workflow, how to inspect the execution logs with `nextflow log`, and how to delete older work directories with `nextflow clean`.
+
+### 3.1. Re-launch a workflow with `-resume`
+
+Sometimes, you're going to want to re-run a pipeline that you've already launched previously without redoing any work that was already completed successfully.
+
+Nextflow has an option called `-resume` that allows you to do this.
+Specifically, in this mode, any processes that have already been run with the exact same code, settings and inputs will be skipped.
+This means Nextflow will only run processes that you've added or modified since the last run, or to which you're providing new settings or inputs.
+
+There are two key advantages to doing this:
+
+- If you're in the middle of developing a pipeline, you can iterate more rapidly since you only have to run the process(es) you're actively working on in order to test your changes.
+- If you're running a pipeline in production and something goes wrong, in many cases you can fix the issue and relaunch the pipeline, and it will resume running from the point of failure, which can save you a lot of time and compute.
+
+To use it, simply add `-resume` to your command and run it:
+
+```bash
+nextflow run 1-hello.nf --greeting 'Hello World!' -resume
+```
+
+The console output should look similar.
+
+
+ Command output
+
+```console linenums="1"
+ N E X T F L O W ~ version 25.04.3
+
+Launching `1-hello.nf` [tiny_noyce] DSL2 - revision: c33d41f479
+
+[a3/7be2fa] process > sayHello [100%] 1 of 1, cached: 1 ✔
+```
+
+
+
+Look for the `cached:` bit that has been added in the process status line (line 5), which means that Nextflow has recognized that it has already done this work and simply reused the result from the previous successful run.
+
+You can also see that the work subdirectory hash is the same as in the previous run.
+Nextflow is literally pointing you to the previous execution and saying "I already did that over there."
+
+!!! Tip
+
+ When your re-run a pipeline with `resume`, Nextflow does not overwrite any files written to a `publishDir` directory by any process call that was previously run successfully.
+
+### 3.2. Inspect the log of past executions
+
+Whenever you launch a nextflow workflow, a line gets written to a log file called `history`, under a hidden directory called `.nextflow` in the current working directory.
+
+
+ File contents
+
+```txt title=".nextflow/history" linenums="1"
+2025-07-04 19:27:09 1.8s wise_watson OK 3539118582ccde68dde471cc2c66295c a02c9c46-c3c7-4085-9139-d1b9b5b194c8 nextflow run 1-hello.nf --greeting 'Hello World'
+2025-07-04 19:27:20 2.9s spontaneous_blackwell OK 3539118582ccde68dde471cc2c66295c 59a5db23-d83c-4c02-a54e-37ddb73a337e nextflow run 1-hello.nf --greeting Bonjour
+2025-07-04 19:27:31 1.8s gigantic_yonath OK 3539118582ccde68dde471cc2c66295c 5acaa83a-6ad6-4509-bebc-cb25d5d7ddd0 nextflow run 1-hello.nf --greeting 'Dobry den'
+2025-07-04 19:27:45 2.4s backstabbing_swartz OK 3539118582ccde68dde471cc2c66295c 5f4b3269-5b53-404a-956c-cac915fbb74e nextflow run 1-hello.nf --greeting Konnichiwa
+2025-07-04 19:27:57 2.1s goofy_wilson OK 3539118582ccde68dde471cc2c66295c 5f4b3269-5b53-404a-956c-cac915fbb74e nextflow run 1-hello.nf --greeting Konnichiwa -resume
+```
+
+
+
+This file gives you the timestamp, run name, status, revision ID, session ID and full command line for every Nextflow run that has been launched from within the current working directory.
+
+A more convenient way to access this information is to use the `nextflow log` command.
+
+```bash
+nextflow log
+```
+
+This will output the contents of the log file to the terminal, augmented with a header line.
+
+
+ Command output
+
+```console linenums="1"
+TIMESTAMP DURATION RUN NAME STATUS REVISION ID SESSION ID COMMAND
+2025-07-04 19:27:09 1.8s wise_watson OK 3539118582 a02c9c46-c3c7-4085-9139-d1b9b5b194c8 nextflow run 1-hello.nf --greeting 'Hello World'
+2025-07-04 19:27:20 2.9s spontaneous_blackwell OK 3539118582 59a5db23-d83c-4c02-a54e-37ddb73a337e nextflow run 1-hello.nf --greeting Bonjour
+2025-07-04 19:27:31 1.8s gigantic_yonath OK 3539118582 5acaa83a-6ad6-4509-bebc-cb25d5d7ddd0 nextflow run 1-hello.nf --greeting 'Dobry den'
+2025-07-04 19:27:45 2.4s backstabbing_swartz OK 3539118582 5f4b3269-5b53-404a-956c-cac915fbb74e nextflow run 1-hello.nf --greeting Konnichiwa
+2025-07-04 19:27:57 2.1s goofy_wilson OK 3539118582 5f4b3269-5b53-404a-956c-cac915fbb74e nextflow run 1-hello.nf --greeting Konnichiwa -resume
+```
+
+
+
+You'll notice that the session ID changes whenever you run a new `nextflow run` command, EXCEPT if you're using the `-resume` option.
+In that case, the session ID stays the same.
+
+Nextflow uses the session ID to group run caching information under the `cache` directory, also located under `.nextflow`.
+
+### 3.3. Delete older work directories
+
+During the development process, you'll typically run your draft pipelines a large number of times, which can lead to an accumulation of very many files across many subdirectories.
+Since the subdirectories are named randomly, it is difficult to tell from their names what are older vs. more recent runs.
+
+Nextflow includes a convenient `clean` subcommand that can automatically delete the work subdirectories for past runs that you no longer care about, with several [options](https://www.nextflow.io/docs/latest/reference/cli.html#clean) to control what will be deleted.
+
+Here we show you an example that deletes all subdirectories from runs before a given run, specified using its run name.
+The run name is the machine-generated two-part string shown in square brackets in the `Launching (...)` console output line, which we also saw recorded in the Nextflow log that we looked at earlier.
+
+You can use the Nextflow log to look up a run based on its timestamp and/or command line.
+
+Once we have that, first we try the `nextflow clean` command using the dry run flag `-n` to check what will be deleted:
+
+```bash
+nextflow clean -before backstabbing_swartz -n
+```
+
+The output will have different task directory names and may have a different number of lines, but it should look similar to the example given below.
+
+
+ Command output
+
+```console title="Output"
+Would remove /workspaces/training/hello-nextflow/work/eb/1a5de36637b475afd88fca7f79e024
+Would remove /workspaces/training/hello-nextflow/work/6b/19b0e002ea13486d3a0344c336c1d0
+Would remove /workspaces/training/hello-nextflow/work/45/9a6dd7ab771f93003d040956282883
+```
+
+
+
+If you don't see any lines output, you either did not provide a valid run name or there are no past runs to delete.
+
+If the output looks as expected and you want to proceed with the deletion, re-run the command with the `-f` flag instead of `-n`:
+
+```bash
+nextflow clean -before backstabbing_swartz -f
+```
+
+The output should be similar to before, but now saying 'Removed' instead of 'Would remove'.
+
+
+ Command output
+
+```console title="Output"
+Removed /workspaces/training/hello-nextflow/work/eb/1a5de36637b475afd88fca7f79e024
+Removed /workspaces/training/hello-nextflow/work/6b/19b0e002ea13486d3a0344c336c1d0
+Removed /workspaces/training/hello-nextflow/work/45/9a6dd7ab771f93003d040956282883
+```
+
+
+
+!!! Warning
+
+ Deleting work subdirectories from past runs removes them from Nextflow's cache and deletes any outputs that were stored in those directories.
+ That means it breaks Nextflow's ability to resume execution without re-running the corresponding processes.
+
+ You are responsible for saving any outputs that you care about or plan to rely on! If you're using the `publishDir` directive for that purpose, make sure to use the `copy` mode, not the `symlink` mode.
+
+### Takeaway
+
+You know how to relaunch a pipeline without repeating steps that were already run in an identical way, inspect the execution log, and use the `nextflow clean` command to clean up old work directories.
+
+### What's next?
+
+Take a little break! You've just absorbed the building blocks of Nextflow syntax and basic usage instructions.
+
+In the next section of this training, we're going to look at four successively more realistic versions of the Hello World pipeline that will demonstrate how Nextflow allows you to process multiple inputs efficiently, run workflows composed of multiple steps connected together, leverage modular code components, and utilize containers for greater reproducibility and portability.
diff --git a/docs/nextflow_run/01_orientation.md b/docs/nextflow_run/01_orientation.md
deleted file mode 100644
index 1be4247f1..000000000
--- a/docs/nextflow_run/01_orientation.md
+++ /dev/null
@@ -1,46 +0,0 @@
-# Orientation
-
-The GitHub Codespaces environment contains all the software, code and data necessary to work through this training course, so you don't need to install anything yourself.
-However, you do need a (free) account to log in, and you should take a few minutes to familiarize yourself with the interface.
-
-If you have not yet done so, please follow [this link](../../envsetup/) before going any further.
-
-## Materials provided
-
-Throughout this training course, we'll be working in the `run-nextflow/` directory.
-This directory contains all the code files, test data and accessory files you will need.
-
-Feel free to explore the contents of this directory; the easiest way to do so is to use the file explorer on the left-hand side of the GitHub Codespaces workspace.
-Alternatively, you can use the `tree` command.
-Throughout the course, we use the output of `tree` to represent directory structure and contents in a readable form, sometimes with minor modifications for clarity.
-
-Here we generate a table of contents to the second level down:
-
-```bash
-tree . -L 2
-```
-
-If you run this inside `run-nextflow`, you should see the following output: [TODO]
-
-```console title="Directory contents"
-.
-```
-
-!!!note
-
- Don't worry if this seems like a lot; we'll go through the relevant pieces at each step of the course.
- This is just meant to give you an overview.
-
-**Here's a summary of what you should know to get started:**
-
-[TODO]
-
-!!!tip
-
- If for whatever reason you move out of this directory, you can always run this command to return to it:
-
- ```bash
- cd /workspaces/training/run-nextflow
- ```
-
-Now, to begin the course, click on the arrow in the bottom right corner of this page.
diff --git a/docs/nextflow_run/02_pipeline.md b/docs/nextflow_run/02_pipeline.md
new file mode 100644
index 000000000..2d193e516
--- /dev/null
+++ b/docs/nextflow_run/02_pipeline.md
@@ -0,0 +1,1168 @@
+# Part 2: Run pipelines
+
+In Part 1 of this course (Run Basic Operations), we started with an example workflow that had only minimal features in order to keep the code complexity low.
+For example, `1-hello.nf` used a command-line parameter (`--greeting`) to provide a single value at a time.
+
+However, most real-world pipelines use more sophisticated features in order to enable efficient processing of large amounts of data at scale, and apply multiple processing steps chained together by sometimes complex logic.
+
+In this part of the training, we demonstrate key features of real-world pipelines by trying out expanded versions of the original Hello World pipeline.
+
+## 1. Processing input data from a file
+
+In a real-world pipeline, we typically want to process multiple data points (or data series) contained in one or more input files.
+And wherever possible, we want to run the processing of independent data in parallel, to shorten the time spent waiting for analysis.
+
+To enable this efficiently, Nextflow uses a system of queues called **channels**.
+
+To demonstrate this, we've prepared a a CSV file called `greetings.csv` that contains several input greetings, mimicking the kind of columnar data you might want to process in a real data analysis.
+
+
+ CSV file contents
+
+```csv title="greetings.csv" linenums="1"
+Hello,English,123
+Bonjour,French,456
+Holà,Spanish,789
+```
+
+Note that the numbers are not meaningful, they are just there for illustrative purposes.
+
+
+
+And we've written an improved version of the original workflow, now called `2a-inputs.nf`, that will read in the CSV file, extract the greetings and write each of them to a separate file.
+
+
+--8<-- "docs/nextflow_run/img/hello-pipeline-multi-inputs.svg"
+
+
+Let's run the workflow first, and we'll take a look at the relevant Nextflow code afterward.
+
+### 1.1. Run the workflow
+
+Run the following command in your terminal.
+
+```bash
+nextflow run 2a-inputs.nf --input greetings.csv
+```
+
+This should run without error.
+
+
+ Command output
+
+```console linenums="1"
+ N E X T F L O W ~ version 25.04.3
+
+Launching `2a-inputs.nf` [mighty_sammet] DSL2 - revision: 29fb5352b3
+
+executor > local (3)
+[8e/0eb066] sayHello (2) [100%] 3 of 3 ✔
+```
+
+
+
+Excitingly, this seems to indicate that '3 of 3' calls were made for the process, which is encouraging, since there were three rows of data in the CSV we provided as input.
+This suggests the `sayHello()` process was called three times, once on each input row.
+
+### 1.2. Find the outputs in the `results` directory
+
+Let's look at the 'results' directory to see if our workflow is still writing a copy of our outputs there.
+
+
+ Directory contents
+
+```console title="results/" linenums="1"
+results
+├── Bonjour-output.txt
+├── Hello-output.txt
+└── Holà-output.txt
+```
+
+
+
+Yes! We see three output files with different names, conveniently enough.
+(Spoiler: we changed the workflow to name the files differently.)
+
+If you haven't deleted the `results` folder when running Part 1 of this training, you'll see the `output.txt` file in there too.
+
+You can open each of them to satisfy yourself that they contain the appropriate greeting string.
+
+
+ File contents
+
+```console title="results/Hello-output.txt"
+Hello
+```
+
+```console title="results/Bonjour-output.txt"
+Bonjour
+```
+
+```console title="results/Holà-output.txt"
+Holà
+```
+
+
+
+This confirms each greeting in the input file has been processed appropriately.
+
+### 1.3. Find the original outputs and logs
+
+You may have noticed that the console output above referred to only one task directory.
+Does that mean all three calls to `sayHello()` were executed within that one task directory?
+
+Let's have a look inside that `8e/0eb066` task directory:
+
+
+ Directory contents
+```console title="8e/0eb066"
+work/8e/0eb066071cdb4123906b7b4ea8b047/
+└── Bonjour-output.txt
+```
+
+
+No! We only find the output corresponding to one of the greetings (as well as the accessory files if we enable display of hidden files).
+
+So what's going on here?
+
+By default, the ANSI logging system writes the status information for all calls to the same process on the same line.
+As a result, it only showed us one of the three task directory paths (`8e/0eb066`) in the console output.
+There are two others that are not listed there.
+
+We can modify the logging behavior to see the full list of process calls by adding the `-ansi-log false` to the command as follows:
+
+```bash
+nextflow run 2a-inputs.nf --input greetings.csv -ansi-log false
+```
+
+This time we see all three process runs and their associated work subdirectories listed in the output.
+
+
+ Command output
+
+```console linenums="1"
+N E X T F L O W ~ version 25.04.3
+Launching `2a-inputs.nf` [pedantic_hamilton] DSL2 - revision: 6bbc42e49f
+[ab/1a8ece] Submitted process > sayHello (1)
+[0d/2cae24] Submitted process > sayHello (2)
+[b5/0df1d6] Submitted process > sayHello (3)
+```
+
+Notice that the way the status is reported is a bit different between the two logging modes.
+In the condensed mode, Nextflow reports whether calls were completed successfully or not.
+In this expanded mode, it only reports that they were submitted.
+
+
+
+This confirms that the `sayHello()` process gets called three times, and a separate task directory is created for each one.
+
+If we look inside each of the task directories listed there, we can verify that each one corresponds to one of the greetings.
+
+
+ Directory contents
+
+```console title="ab/1a8ece"
+work/ab/1a8ece307e53f03fce689dde904b64/
+└── Hello-output.txt
+```
+
+```console title="0d/2cae24"
+work/0d/2cae2481a53593bc607077c80c9466/
+└── Bonjour-output.txt
+```
+
+```console title="b5/0df1d6"
+work/b5/0df1d642353269909c2ce23fc2a8fa/
+└── Holà-output.txt
+```
+
+
+
+This confirms that each process call is executed in isolation from all the others.
+That has many advantages, including avoiding collisions if the process produces any intermediate files with non-unique names.
+
+!!! Tip
+
+ For a complex workflow, or a large number of inputs, having the full list output to the terminal might get a bit overwhelming, so you might prefer not to use `-ansi-log false` in those cases.
+
+### 1.4. Examine the code
+
+So this version of the workflow is capable of reading in a CSV file of inputs, processing the inputs separately, and naming the outputs uniquely.
+
+Let's take a look at what makes that possible in the workflow code.
+Once again, we're not aiming to memorize code syntax, but to identify signature components of the workflow that provide important functionality.
+
+
+ Code
+
+```groovy title="2a-inputs.nf" linenums="1"
+#!/usr/bin/env nextflow
+
+/*
+ * Use echo to print 'Hello World!' to a file
+ */
+process sayHello {
+
+ publishDir 'results', mode: 'copy'
+
+ input:
+ val greeting
+
+ output:
+ path "${greeting}-output.txt"
+
+ script:
+ """
+ echo '$greeting' > '$greeting-output.txt'
+ """
+}
+
+workflow {
+
+ // create a channel for inputs from a CSV file
+ greeting_ch = Channel.fromPath(params.input)
+ .splitCsv()
+ .map { line -> line[0] }
+
+ // emit a greeting
+ sayHello(greeting_ch)
+}
+```
+
+
+
+#### 1.4.1. Loading the input data from the CSV
+
+This is the most interesting part: how did we switch from taking a single value from the command-line, to taking a CSV file, parsing it and processing the individual greetings it contains?
+
+In Nextflow, we do that with a **channel**: a construct designed to handle inputs efficiently and shuttle them from one step to another in multi-step workflows, while providing built-in parallelism and many additional benefits.
+
+Let's break it down.
+
+```groovy title="2a-inputs.nf" linenums="22"
+workflow {
+
+ // create a channel for inputs from a CSV file
+ greeting_ch = Channel.fromPath(params.input)
+ .splitCsv()
+ .map { line -> line[0] }
+```
+
+This is where the magic happens, starting at line 25.
+Here's what that line means in plain English:
+
+- `Channel` creates a **channel**, i.e. a queue that will hold the data
+- `.fromPath` specifies the data source is a filepath
+- `(params.input)` specifies the filepath is provided by `--input` on the command line
+
+In other words, that line tells Nextflow: take the filepath given with `--input` and get ready to treat its contents as input data.
+
+Then the next two lines apply **operators** that do the actual parsing of the file and loading of the data into the appropriate data structure:
+
+- `.splitCsv()` tells Nextflow to parse the CSV file into an array representing rows and columns
+- `.map { line -> line[0] }` tells Nextflow to take only the element in the first column from each row
+
+So in practice, starting from the following CSV file:
+
+```csv title="greetings.csv" linenums="1"
+Hello,English,123
+Bonjour,French,456
+Holà,Spanish,789
+```
+
+We have transformed that into an array that looks like this:
+
+```txt title="Array contents"
+[[Hello,English,123],[Bonjour,French,456],[Holà,Spanish,789]]
+```
+
+And then we've taken the first element from each of the three rows and loaded them into a Nextflow channel that now contains: `Hello`, `Bonjour`, and `Holà`.
+
+The result of this very short snippet of code is a channel called `greeting_ch` loaded with the three individual greetings from the CSV file, ready for processing.
+
+#### 1.4.2. Call the process on each greeting
+
+Next, in the last line of the workflow block, we provide the loaded `greeting_ch` channel as input to the `sayHello()` process.
+
+```groovy title="2a-inputs.nf" linenums="28"
+ sayHello(greeting_ch)
+}
+```
+
+This tells Nextflow to run the process _individually_ on each element in the channel, i.e. on each greeting.
+
+And because Nextflow is smart like that, it will run these process calls in parallel if possible, depending on the available computing infrastructure.
+
+That is how you can achieve efficient and scalable processing of a lot of data (many samples, or data points, whatever is your unit of research) with comparatively very little code.
+
+#### 1.4.3. Ensure the outputs are uniquely named
+
+Finally, it's worth taking a quick look at how we get the output files to be named uniquely.
+
+```groovy title="2a-inputs.nf" linenums="13"
+ output:
+ path "${greeting}-output.txt"
+
+ script:
+ """
+ echo '$greeting' > '$greeting-output.txt'
+ """
+```
+
+You see that, compared to the version of this process in `1-hello.nf`, the output declaration and the relevant bit of the command have changed to include the greeting value in the output file name.
+
+This is one way to ensure that the output file names won't collide when they get published to the common `results` directory.
+
+And that's the only change we've had to make inside the process declaration.
+
+### Takeaway
+
+You understand at a basic level how channels and operators enable us to process multiple inputs efficiently.
+
+### What's next?
+
+Discover how multi-step workflows are constructed and how they operate.
+
+---
+
+## 2. Running multi-step workflows
+
+Most real-world workflows involve more than one step.
+Let's build on what we just learned about channels, and look at how Nextflow uses channels and operators to connect processes together in a multi-step workflow.
+
+To that end, we provide you with an example workflow that chains together three separate steps and demonstrates the following:
+
+1. Making data flow from one process to the next
+2. Collecting outputs from multiple process calls into a single process call
+
+Specifically, we made an expanded version of the workflow called `2b-multistep.nf` that takes each input greeting, converts it to uppercase, then collects all the uppercased greetings into a single output file.
+
+
+--8<-- "docs/nextflow_run/img/hello-pipeline-multi-steps.svg"
+
+
+As previously, we'll run the workflow first then look at the code to see what's changed.
+
+### 2.1. Run the workflow
+
+Run the following command in your terminal:
+
+```bash
+nextflow run 2b-multistep.nf --input greetings.csv
+```
+
+Once again this should run successfully.
+
+
+ Command output
+
+```console linenums="1"
+ N E X T F L O W ~ version 25.04.3
+
+Launching `2b-multistep.nf` [soggy_franklin] DSL2 - revision: bc8e1b2726
+
+[d6/cdf466] sayHello (1) | 3 of 3 ✔
+[99/79394f] convertToUpper (2) | 3 of 3 ✔
+[1e/83586c] collectGreetings | 1 of 1 ✔
+```
+
+
+
+You see that as promised, multiple steps were run as part of the workflow; the first two (`sayHello` and `convertToUpper`) were presumably run on each individual greeting, and the third (`collectGreetings`) will have been run only once, on the outputs of all three of the `convertToUpper` calls.
+
+### 2.2. Find the outputs
+
+Let's verify that that is in fact what happened by taking a look in the `results` directory.
+
+
+ Directory contents
+
+```console title="Directory contents"
+results
+├── Bonjour-output.txt
+├── COLLECTED-output.txt
+├── Hello-output.txt
+├── Holà-output.txt
+├── UPPER-Bonjour-output.txt
+├── UPPER-Hello-output.txt
+└── UPPER-Holà-output.txt
+```
+
+
+
+Look at the file names and check their contents to confirm that they are what you expect; for example:
+
+```console title="bash"
+cat results/COLLECTED-output.txt
+```
+
+
+ Command output
+
+```console
+HELLO
+BONJOUR
+HOLà
+```
+
+
+
+That is the expected final result of our multi-step pipeline.
+
+### 2.3. Examine the code
+
+Let's look at the code and see what we can tie back to what we just observed.
+
+
+ Code
+
+```groovy title="2a-inputs.nf" linenums="1"
+#!/usr/bin/env nextflow
+
+/*
+ * Use echo to print 'Hello World!' to a file
+ */
+process sayHello {
+
+ publishDir 'results', mode: 'copy'
+
+ input:
+ val greeting
+
+ output:
+ path "${greeting}-output.txt"
+
+ script:
+ """
+ echo '$greeting' > '$greeting-output.txt'
+ """
+}
+
+/*
+ * Use a text replacement tool to convert the greeting to uppercase
+ */
+process convertToUpper {
+
+ publishDir 'results', mode: 'copy'
+
+ input:
+ path input_file
+
+ output:
+ path "UPPER-${input_file}"
+
+ script:
+ """
+ cat '$input_file' | tr '[a-z]' '[A-Z]' > 'UPPER-${input_file}'
+ """
+}
+
+/*
+ * Collect uppercase greetings into a single output file
+ */
+process collectGreetings {
+
+ publishDir 'results', mode: 'copy'
+
+ input:
+ path input_files
+
+ output:
+ path "COLLECTED-output.txt"
+
+ script:
+ """
+ cat ${input_files} > 'COLLECTED-output.txt'
+ """
+}
+
+workflow {
+
+ // create a channel for inputs from a CSV file
+ greeting_ch = Channel.fromPath(params.input)
+ .splitCsv()
+ .map { line -> line[0] }
+
+ // emit a greeting
+ sayHello(greeting_ch)
+
+ // convert the greeting to uppercase
+ convertToUpper(sayHello.out)
+
+ // collect all the greetings into one file
+ collectGreetings(convertToUpper.out.collect())
+}
+
+```
+
+
+
+The most obvious difference compared to the previous version of the workflow is that now there are multiple process definitions, and correspondingly, several process calls in the workflow block.
+
+#### 2.3.1. Multiple process definitions
+
+In addition to the original `sayHello` process, we now also have `convertToUpper` and `collectGreetings`, which match the names of the processes we saw in the console output.
+
+All three are structured in the same way and follow roughly the same logic.
+We won't go into that in detail, but it shows how a process can be given additional parameters and emit multiple outputs.
+
+#### 2.3.2. Processes are connected via channels
+
+The really interesting thing to look at here is how the process calls are chained together in the workflow block.
+
+
+ Code
+
+```groovy title="2a-inputs.nf" linenums="69"
+workflow {
+
+ // create a channel for inputs from a CSV file
+ greeting_ch = Channel.fromPath(params.input)
+ .splitCsv()
+ .map { line -> line[0] }
+
+ // emit a greeting
+ sayHello(greeting_ch)
+
+ // convert the greeting to uppercase
+ convertToUpper(sayHello.out)
+
+ // collect all the greetings into one file
+ collectGreetings(convertToUpper.out.collect())
+}
+```
+
+
+
+You can see that the first process call, `sayHello(greeting_ch)`, is unchanged.
+
+Then the next process call, to `convertToUpper`, _refers_ to the output of `sayHello` as `sayHello.out`.
+
+```groovy title="2a-inputs.nf" linenums="79"
+ // convert the greeting to uppercase
+ convertToUpper(sayHello.out)
+```
+
+This tells Nextflow to provide `sayHello.out`, which represents the channel output by `sayHello()`, as an input to `convertToUpper`.
+
+That is, at its simplest, how we shuttle data from one step to the next in Nextflow.
+
+Finally, the third call, `collectGreetings`, is doing the same thing, with a twist:
+
+```groovy title="2a-inputs.nf" linenums="82"
+ // collect all the greetings into one file
+ collectGreetings(convertToUpper.out.collect())
+```
+
+This one is a bit more complicated and deserves its own discussion.
+
+#### 2.3.3. Operators provide additional wiring options
+
+What we're seeing in `convertToUpper.out.collect()` is the use of another operator (like `splitCsv` and `map` in the previous section), called `collect`.
+
+This operator is used to collect the outputs from multiple calls to the same process (as when we run `sayHello` on multiple greetings independently) and package them into a single channel element.
+
+This allows us to take all the separate uppercased greetings produced by the second step of the workflow and feed them all together to a single call in the third step of the pipeline.
+
+
+--8<-- "docs/nextflow_run/img/with-collect-operator.svg"
+
+
+If we didn't apply `collect()` to the output of `convertToUpper()` before feeding it to `collectGreetings()`, Nextflow would simply run `collectGreetings()` independently on each greeting, which would not achieve our goal.
+
+
+--8<-- "docs/nextflow_run/img/without-collect-operator.svg"
+
+
+There are many other [operators](https://www.nextflow.io/docs/latest/reference/operator.html#operator-page) available to apply transformations to the contents of channels between process calls.
+
+This gives pipeline developers a lot of flexibility for customizing the flow logic of their pipeline.
+The downside is that it can sometimes make it harder to decipher what the pipeline is doing.
+
+### 2.4. Use the graph preview
+
+One very helpful tool for understanding what a pipeline does, if it's not adequately documented, is the graph preview functionality available in VSCode thanks to the Nextflow extension. You can see this in the training environment by clicking on the small `DAG preview` link displayed just above the workflow block in any Nextflow script.
+
+
+
+This does not show operators, but it does give a useful representation of how process calls are connected and what are their inputs.
+
+### Takeaway
+
+You understand at a basic level how multi-step workflows are constructed using channels and operators and how they operate.
+
+### What's next?
+
+Learn how Nextflow pipelines can be modularized to promote code reuse and maintainability.
+
+---
+
+## 3. Running modularized pipelines
+
+So far, all the workflows we've looked at have consisted of one single workflow file containing all the relevant code.
+
+However, real-world pipelines typically benefit from being _modularized_, meaning that the code is split into different files.
+This can make their development and maintenance more efficient and sustainable.
+
+
+ --8<-- "docs/side_quests/img/nf-core/nested.excalidraw.svg"
+
+
+Here we are going to demonstrate the most common form of code modularity in Nextflow, which is the use of **modules**.
+
+In Nextflow, a **module** is a single process definition that is encapsulated by itself in a standalone code file.
+To use a module in a workflow, you just add a single-line import statement to your workflow code file; then you can integrate the process into the workflow the same way you normally would.
+
+Putting processes into individual modules makes it possible to reuse process definitions in multiple workflows without producing multiple copies of the code.
+This makes the code more shareable, flexible and maintainable.
+
+We have of course once again prepared a suitable workflow for demonstration purposes, called `2c-modules.nf`, along with a set of modules located in the `modules/` directory.
+
+
+ Directory contents
+
+```console title="modules/"
+modules/
+├── collectGreetings.nf
+├── convertToUpper.nf
+├── cowpy.nf
+└── sayHello.nf
+```
+
+
+
+You see there are four Nextflow files, each named after one of the processes.
+You can ignore the `cowpy.nf` file for now; we'll get to that one later.
+
+### 3.1. Examine the code
+
+This time we're going to look at the code first, so let's open each of the files listed above (not shown here).
+
+We see that the code for the processes and workflow logic are exactly the same as in the previous version of the workflow.
+However, the process code is now located in the modules instead of being in the main workflow file, and there are now import statements in the workflow file telling Nextflow to pull them in at runtime.
+
+```groovy title="hello-modules.nf" linenums="9" hl_lines="4"
+// Include modules
+include { sayHello } from './modules/sayHello.nf'
+include { convertToUpper } from './modules/convertToUpper.nf'
+include { collectGreetings } from './modules/collectGreetings.nf'
+
+workflow {
+```
+
+You can look inside one of the modules to satisfy yourself that the process definition is unchanged; it's literally just been copy-pasted into a standalone file.
+
+
+ Example: sayHello process module
+
+```groovy title="modules/sayHello.nf" linenums="1"
+#!/usr/bin/env nextflow
+
+/*
+ * Use echo to print 'Hello World!' to a file
+ */
+process sayHello {
+
+ publishDir 'results', mode: 'copy'
+
+ input:
+ val greeting
+
+ output:
+ path "${greeting}-output.txt"
+
+ script:
+ """
+ echo '$greeting' > '$greeting-output.txt'
+ """
+}
+```
+
+
+
+So let's see what it looks like to run this new version.
+
+### 3.2. Run the workflow
+
+Run this command in your terminal, with the `-resume` flag:
+
+```bash
+nextflow run 2c-modules.nf --input greetings.csv -resume
+```
+
+Once again this should run successfully.
+
+
+ Command output
+
+```console linenums="1"
+ N E X T F L O W ~ version 25.04.3
+
+Launching `2c-modules.nf` [soggy_franklin] DSL2 - revision: bc8e1b2726
+
+[j6/cdfa66] sayHello (1) | 3 of 3, cached: ✔
+[95/79484f] convertToUpper (2) | 3 of 3, cached: ✔
+[5e/4358gc] collectGreetings | 1 of 1, cached: ✔
+```
+
+
+
+You'll notice that the process executions all cached successfully, meaning that Nextflow recognized that it has already done the requested work, even though the code has been split up and the main workflow file has been renamed.
+
+None of that matters to Nextflow; what matters is the job script that is generated once all the code has been pulled together and evaluated.
+
+!!! Tip
+
+ It is also possible to encapsulate a section of a workflow as a 'subworkflow' that can be imported into a larger pipeline, but that is outside the scope of this course.
+
+ You can learn more about developing composable workflows in the Side Quest on [Workflows of Workflows](https://training.nextflow.io/latest/side_quests/workflows_of_workflows/).
+
+### Takeaway
+
+You know how processes can be stored in standalone modules to promote code reuse and improve maintainability.
+
+### What's next?
+
+Learn to use containers for managing software dependencies.
+
+---
+
+## 4. Using containerized software
+
+So far the workflows we've been using as examples just needed to run very basic text processing operations using UNIX tools available in our environment.
+
+However, real-world pipelines typically require specialized tools and packages that are not included by default in most environments.
+Usually, you'd need to install these tools, manage their dependencies, and resolve any conflicts.
+
+That is all very tedious and annoying.
+A much better way to address this problem is to use **containers**.
+
+A **container** is a lightweight, standalone, executable unit of software created from a container **image** that includes everything needed to run an application including code, system libraries and settings.
+
+!!! Tip
+
+ We teach this using the technology [Docker](https://www.docker.com/get-started/), but Nextflow supports [several other container technologies](https://www.nextflow.io/docs/latest/container.html#) as well.
+
+### 4.1. Use a container directly
+
+First, let's try interacting with a container directly.
+This will help solidify your understanding of what containers are before we start using them in Nextflow.
+
+#### 4.1.1. Pull the container image
+
+To use a container, you usually download or "pull" a container image from a container registry, and then run the container image to create a container instance.
+
+The general syntax is as follows:
+
+```bash title="Syntax"
+docker pull ''
+```
+
+- `docker pull` is the instruction to the container system to pull a container image from a repository.
+- `''` is the URI address of the container image.
+
+As an example, let's pull a container image that contains [cowpy](https://github.com/jeffbuttars/cowpy), a python implementation of a tool called `cowsay` that generates ASCII art to display arbitrary text inputs in a fun way.
+
+There are various repositories where you can find published containers.
+We used the [Seqera Containers](https://seqera.io/containers/) service to generate this Docker container image from the `cowpy` Conda package: `'community.wave.seqera.io/library/cowpy:1.1.5--3db457ae1977a273'`.
+
+Run the complete pull command:
+
+```bash
+docker pull 'community.wave.seqera.io/library/cowpy:1.1.5--3db457ae1977a273'
+```
+
+This tells the system to download the image specified.
+
+
+ Command output
+
+```console linenums="1"
+Unable to find image 'community.wave.seqera.io/library/cowpy:1.1.5--3db457ae1977a273' locally
+131d6a1b707a8e65: Pulling from library/cowpy
+dafa2b0c44d2: Pull complete
+dec6b097362e: Pull complete
+f88da01cff0b: Pull complete
+4f4fb700ef54: Pull complete
+92dc97a3ef36: Pull complete
+403f74b0f85e: Pull complete
+10b8c00c10a5: Pull complete
+17dc7ea432cc: Pull complete
+bb36d6c3110d: Pull complete
+0ea1a16bbe82: Pull complete
+030a47592a0a: Pull complete
+622dd7f15040: Pull complete
+895fb5d0f4df: Pull complete
+Digest: sha256:fa50498b32534d83e0a89bb21fec0c47cc03933ac95c6b6587df82aaa9d68db3
+Status: Downloaded newer image for community.wave.seqera.io/library/cowpy:1.1.5--3db457ae1977a273
+community.wave.seqera.io/library/cowpy:1.1.5--3db457ae1977a273
+```
+
+
+
+Once the download is complete, you have a local copy of the container image.
+
+#### 4.1.2. Spin up the container
+
+Containers can be run as a one-off command, but you can also use them interactively, which gives you a shell prompt inside the container and allows you to play with the command.
+
+The general syntax is as follows:
+
+```bash title="Syntax"
+docker run --rm '' [tool command]
+```
+
+- `docker run --rm ''` is the instruction to the container system to spin up a container instance from a container image and execute a command in it.
+- `--rm` tells the system to shut down the container instance after the command has completed.
+
+
+
+Fully assembled, the container execution command looks like this:
+
+```bash
+docker run --rm -it 'community.wave.seqera.io/library/cowpy:1.1.5--3db457ae1977a273'
+```
+
+Run that command, and you should see your prompt change to something like `(base) root@b645838b3314:/tmp#`, which indicates that you are now inside the container.
+
+You can verify this by running `ls` to list directory contents:
+
+```bash
+ls /
+```
+
+
+ Command output
+
+```console
+bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var
+```
+
+
+
+You observe see that the filesystem inside the container is different from the filesystem on your host system.
+
+!!! Tip
+
+ When you run a container, it is isolated from the host system by default.
+ This means that the container can't access any files on the host system unless you explicitly allow it to do so by specifying that you want to mount a volume as part of the `docker run` command using the following syntax:
+
+ ```bash title="Syntax"
+ -v :
+ ```
+
+ This effectively establishes a tunnel through the container wall that you can use to access that part of your filesystem.
+
+#### 4.1.3. Run the `cowpy` tool
+
+From inside the container, you can run the `cowpy` command directly.
+
+```bash
+cowpy "Hello Containers"
+```
+
+This produces ASCII art of the default cow character (or 'cowacter') with a speech bubble containing the text we specified.
+
+
+ Command output
+
+```console
+ ______________________________________________________
+< Hello Containers >
+ ------------------------------------------------------
+ \ ^__^
+ \ (oo)\_______
+ (__)\ )\/\
+ ||----w |
+ || ||
+```
+
+
+
+Now that you have tested the basic usage, you can try giving it some parameters.
+For example, the tool documentation says we can set the character with `-c`.
+
+```bash
+cowpy "Hello Containers" -c tux
+```
+
+This time the ASCII art output shows the Linux penguin, Tux, because we specified the `-c tux` parameter.
+
+
+ Command output
+
+```console
+ __________________
+< Hello Containers >
+ ------------------
+ \
+ \
+ .--.
+ |o_o |
+ |:_/ |
+ // \ \
+ (| | )
+ /'\_ _/`\
+ \___)=(___/
+```
+
+
+
+Since you're inside the container, you can run the cowpy command as many times as you like, varying the input parameters, without having to worry about install any libraries on your system itself.
+
+!!! Tip
+
+ Use the '-c' flag to pick a different character, including:
+ `beavis`, `cheese`, `daemon`, `dragonandcow`, `ghostbusters`, `kitty`, `moose`, `milk`, `stegosaurus`, `turkey`, `turtle`, `tux`
+
+Feel free to play around with this.
+When you're done, exit the container using the `exit` command:
+
+```bash
+exit
+```
+
+You will find yourself back in your normal shell.
+
+### 4.2. Use a container in a workflow
+
+When we run a pipeline, we want to be able to tell Nextflow what container to use at each step, and importantly, we want it to handle all that work we just did: pull the container, spin it up, run the command and tear the container down when it's done.
+
+Good news: that's exactly what Nextflow is going to do for us.
+We just need to specify a container for each process.
+
+To demonstrate how this work, we made another version of our workflow that runs `cowpy` on the file of collected greetings produced in the third step.
+
+
+--8<-- "docs/nextflow_run/img/hello-pipeline-cowpy.svg"
+
+
+This should output a file containing the ASCII art with the three greetings in the speech bubble.
+
+#### 4.2.1. Examine the code
+
+The workflow is very similar to the previous one, plus the extra step to run `cowpy.
+The differences are highlighted in the code snippet below.
+
+
+ Code
+
+```groovy title="2d-container.nf" linenums="1" hl_lines="7 25 26"
+#!/usr/bin/env nextflow
+
+// Include modules
+include { sayHello } from './modules/sayHello.nf'
+include { convertToUpper } from './modules/convertToUpper.nf'
+include { collectGreetings } from './modules/collectGreetings.nf'
+include { cowpy } from './modules/cowpy.nf'
+
+workflow {
+
+ // create a channel for inputs from a CSV file
+ greeting_ch = Channel.fromPath(params.input)
+ .splitCsv()
+ .map { line -> line[0] }
+
+ // emit a greeting
+ sayHello(greeting_ch)
+
+ // convert the greeting to uppercase
+ convertToUpper(sayHello.out)
+
+ // collect all the greetings into one file
+ collectGreetings(convertToUpper.out.collect())
+
+ // generate ASCII art with cowpy
+ cowpy(collectGreetings.out, params.character)
+}
+```
+
+
+
+You see that this workflow imports a `cowpy` process from a module file, and calls it on the output of the `collectGreetings()` call.
+
+```groovy title="modules/cowpy.nf" linenums="26"
+cowpy(collectGreetings.out, params.character)
+```
+
+The `cowpy` process, which wraps the cowpy command to generate ASCII art, is defined in the `cowpy.nf` module.
+
+
+ Code
+
+```groovy title="modules/cowpy.nf" linenums="1"
+#!/usr/bin/env nextflow
+
+// Generate ASCII art with cowpy
+process cowpy {
+
+ container 'community.wave.seqera.io/library/cowpy:1.1.5--3db457ae1977a273'
+
+ publishDir 'results', mode: 'copy'
+
+ input:
+ path input_file
+ val character
+
+ output:
+ path "cowpy-${input_file}"
+
+ script:
+ """
+ cat $input_file | cowpy -c "$character" > cowpy-${input_file}
+ """
+
+}
+```
+
+
+
+The `cowpy` process requires two inputs: the path to an input file containing the text to put in the speech bubble (`input_file`), and a value for the character variable.
+
+Importantly, it also includes the line `container 'community.wave.seqera.io/library/cowpy:1.1.5--3db457ae1977a273'`, which points to the container URI we used earlier.
+
+#### 4.2.2. Check that Docker is enabled in the configuration
+
+We're going to slightly anticipate Part 3 of this training course by introducing the topic of configuration.
+
+One of the main ways Nextflow offers for configuring workflow execution is to use a `nextflow.config` file.
+When such a file is present in the current directory, Nextflow will automatically load it in and apply any configuration it contains.
+
+To that end, we include a `nextflow.config` file with a single line of code that enables Docker.
+
+```groovy title="nextflow.config" linenums="1"
+docker.enabled = true
+```
+
+You can check that this is indeed set correctly either by opening the file, or by running the `nextflow config` command in the terminal.
+
+```bash
+nextflow config
+```
+
+
+ Command output
+
+```json title="nextflow.config" linenums="1"
+docker {
+ enabled = true
+}
+```
+
+
+
+That tells Nextflow to use Docker for any process that specifies a compatible container.
+
+!!! Tip
+
+ It is possible to enable Docker execution from the command-line, on a per-run basis, using the `-with-docker ` parameter.
+ However, that only allows us to specify one container for the entire workflow, whereas the approach we just showed you allows us to specify a different container per process.
+ This is better for modularity, code maintenance and reproducibility.
+
+#### 4.2.3. Run the workflow
+
+Let's run the workflow with the `-resume` flag, and specify that we want the character to be the turkey.
+
+```bash
+nextflow run 2d-container.nf --input greetings.csv --character turkey -resume
+```
+
+This should work without error.
+
+
+ Command output
+
+```console linenums="1"
+ N E X T F L O W ~ version 25.04.3
+
+Launching `2d-container.nf` [elegant_brattain] DSL2 - revision: 028a841db1
+
+executor > local (1)
+[95/fa0bac] sayHello (3) | 3 of 3, cached: 3 ✔
+[92/32533f] convertToUpper (3) | 3 of 3, cached: 3 ✔
+[aa/e697a2] collectGreetings | 1 of 1, cached: 1 ✔
+[7f/caf718] cowpy | 1 of 1 ✔
+```
+
+
+
+The first three steps cached since we've already run them before, but the `cowpy` process is new so that actually gets run.
+
+You can find the output of the `cowpy` step in the `results` directory.
+
+
+ Output file contents
+
+```console title="results/cowpy-COLLECTED-output.txt"
+ _________
+/ HOLà \
+| HELLO |
+\ BONJOUR /
+ ---------
+ \ ,+*^^*+___+++_
+ \ ,*^^^^ )
+ \ _+* ^**+_
+ \ +^ _ _++*+_+++_, )
+ _+^^*+_ ( ,+*^ ^ \+_ )
+ { ) ( ,( ,_+--+--, ^) ^\
+ { (\@) } f ,( ,+-^ __*_*_ ^^\_ ^\ )
+ {:;-/ (_+*-+^^^^^+*+*<_ _++_)_ ) ) /
+ ( / ( ( ,___ ^*+_+* ) < < \
+ U _/ ) *--< ) ^\-----++__) ) ) )
+ ( ) _(^)^^)) ) )\^^^^^))^*+/ / /
+ ( / (_))_^)) ) ) ))^^^^^))^^^)__/ +^^
+ ( ,/ (^))^)) ) ) ))^^^^^^^))^^) _)
+ *+__+* (_))^) ) ) ))^^^^^^))^^^^^)____*^
+ \ \_)^)_)) ))^^^^^^^^^^))^^^^)
+ (_ ^\__^^^^^^^^^^^^))^^^^^^^)
+ ^\___ ^\__^^^^^^))^^^^^^^^)\\
+ ^^^^^\uuu/^^\uuu/^^^^\^\^\^\^\^\^\^\
+ ___) >____) >___ ^\_\_\_\_\_\_\)
+ ^^^//\\_^^//\\_^ ^(\_\_\_\)
+ ^^^ ^^ ^^^ ^
+```
+
+What a beautiful turkey!
+
+
+
+You see that the character is saying all the greetings, since it ran on the file of collected uppercased greetings.
+
+More to the point, we were able to run this as part of our pipeline without having to do a proper installation of cowpy and all its dependencies.
+And we can now share the pipeline with collaborators and have them run it on their infrastructure without them needing to install anything either, aside from Docker or one of its alternatives (such as Singularity/Apptainer) as mentioned above.
+
+#### 4.2.4. Inspect how Nextflow launched the containerized task
+
+Let's take a look at the `.command.run` file inside the task directory where the `cowpy` call was executed.
+This file contains all the commands Nextflow ran on your behalf in the course of executing the pipeline.
+
+Open the `.command.run` file and search for `nxf_launch` to find the launch command Nextflow used.
+
+
+ Partial file contents
+
+```bash title="work/7f/caf7189fca6c56ba627b75749edcb3/.command.run"
+nxf_launch() {
+ docker run -i --cpu-shares 1024 -e "NXF_TASK_WORKDIR" -v /workspaces/training/hello-nextflow/work:/workspaces/training/hello-nextflow/work -w "$NXF_TASK_WORKDIR" --name $NXF_BOXID community.wave.seqera.io/library/pip_cowpy:131d6a1b707a8e65 /bin/bash -ue /workspaces/training/nextflow-run/work/7f/caf7189fca6c56ba627b75749edcb3/.command.sh
+}
+```
+
+
+
+This launch command shows that Nextflow is using a very similar `docker run` command to launch the process call as we did when we ran it manually.
+It also mounts the corresponding work subdirectory into the container, sets the working directory inside the container accordingly, and runs our templated bash script in the `.command.sh` file.
+
+This confirms that all the hard work we had to do manually in the previous section is now done for us by Nextflow!
+
+### Takeaway
+
+You understand what role containers play in managing software tool versions and ensuring reproducibility.
+
+More generally, you have a basic understanding of what are the core components of real-world Nextflow pipelines and how they are organized.
+You know the fundamentals of how Nextflow can process multiple inputs efficiently, run workflows composed of multiple steps connected together, leverage modular code components, and utilize containers for greater reproducibility and portability.
+
+### What's next?
+
+Take another break! That was a big pile of information about how Nextflow pipelines work.
+In the next section of this training, we're going to delve deeper into the topic of configuration.
+You will learn how to configure the execution of your pipeline to fit your infrastructure as well as manage configuration of inputs and parameters.
diff --git a/docs/nextflow_run/02_run_basics.md b/docs/nextflow_run/02_run_basics.md
deleted file mode 100644
index 14f0b7e56..000000000
--- a/docs/nextflow_run/02_run_basics.md
+++ /dev/null
@@ -1,10 +0,0 @@
-# Part 1: Run Basics
-
-[TODO]
-
-Should cover:
-
-- basic project structure (main.nf, modules, nextflow.config)
-- run from CLI (re-use from Hello-World)
-- basic config elements (refer to hello-config) including profiles
-- running resource profiling and adapting the config
diff --git a/docs/nextflow_run/03_config.md b/docs/nextflow_run/03_config.md
new file mode 100644
index 000000000..751adffb7
--- /dev/null
+++ b/docs/nextflow_run/03_config.md
@@ -0,0 +1,960 @@
+# Part 3: Configuration
+
+This section will explore how to manage the configuration of a Nextflow pipeline in order to customize its behavior, adapt it to different environments, and optimize resource usage _without altering a single line of the workflow code itself_.
+
+There are multiple ways to do this; here we are going to use the simplest and most common configuration file mechanism, the `nextflow.config` file.
+As noted previously, whenever there is a file named `nextflow.config` in the current directory, Nextflow will automatically load configuration from it.
+
+!!! Tip
+
+ Anything you put into the `nextflow.config` can be overridden at runtime by providing the relevant process directives or parameters and values on the command line, or by importing another configuration file, according to the order of precedence described [here](https://www.nextflow.io/docs/latest/config.html).
+
+In this part of the training, we're going to use the `nextflow.config` file to demonstrate essential components of Nextflow configuration such as process directives, executors, profiles, and parameter files.
+By learning to utilize these configuration options effectively, you can enhance the flexibility, scalability, and performance of your pipelines.
+
+To exercise these elements of configuration, we're going to be running a fresh copy of the workflow we last ran at the end of Part 2 of this training course, renamed `3-main.nf`.
+
+---
+
+## 1. Determine what software packaging technology to use
+
+The first step toward adapting your workflow configuration to your compute environment is specifying where the software packages that will get run in each step are going to be coming from.
+Are they already installed in the local compute environment?
+Do we need to retrieve images and run them via a container system?
+Or do we need to retrieve Conda packages and build a local Conda environment?
+
+For most of this training course so far, we just used locally installed software in our workflow.
+Then in the last section of Part 2, we introduced Docker containers and the `nextflow.config` file, which we used to enable the use of Docker containers.
+
+Now let's see how we can configure an alternative software packaging option via the `nextflow.config` file.
+
+### 1.1. Disable Docker and enable Conda in the config file
+
+Let's pretend we're working on an HPC cluster and the admin doesn't allow the use of Docker for security reasons.
+
+Fortunately for us, Nextflow supports multiple other container technologies such as including Singularity/Apptainer (which is more widely used on HPC), and software package managers such as Conda.
+
+We can change our configuration file to use Conda instead of Docker.
+To do so, we switch the value of `docker.enabled` to `false`, and add a directive enabling the use of Conda:
+
+=== "After"
+
+ ```groovy title="nextflow.config" linenums="1" hl_lines="1-2"
+ docker.enabled = false
+ conda.enabled = true
+ ```
+
+=== "Before"
+
+ ```groovy title="nextflow.config" linenums="1"
+
+ docker.enabled = true
+ ```
+
+This will allow Nextflow to create and utilize Conda environments for processes that have Conda packages specified.
+Which means we now need to add one of those to the `cowpy` process definition!
+
+### 1.2. Specify a Conda package in the process definition
+
+We've already retrieved the URI for a Conda package containing the `cowpy` tool: `conda-forge::cowpy==1.1.5`
+
+!!! Tip
+
+ There are a few different ways to get the URI for a given conda package.
+ We recommend using the [Seqera Containers](https://seqera.io/containers/) search query, which will give you a URI that you can copy and paste, even if you're not planning to create a container from it.
+
+Now we add the URI to the `cowpy` process definition using the `conda` directive:
+
+=== "After"
+
+ ```console title="modules/cowpy.nf" linenums="4" hl_lines="4"
+ process cowpy {
+
+ container 'community.wave.seqera.io/library/cowpy:1.1.5--3db457ae1977a273'
+ conda 'conda-forge::cowpy==1.1.5'
+
+ publishDir 'results', mode: 'copy'
+ ```
+
+=== "Before"
+
+ ```console title="modules/cowpy.nf" linenums="4"
+ process cowpy {
+
+ container 'community.wave.seqera.io/library/cowpy:1.1.5--3db457ae1977a273'
+
+ publishDir 'results', mode: 'copy'
+ ```
+
+To be clear, we're not _replacing_ the `container` directive, we're _adding_ an alternative option.
+
+### 1.3. Run the workflow to verify that it can use Conda
+
+Let's try it out.
+
+```bash
+nextflow run 3-main.nf --input greetings.csv --character turkey
+```
+
+This should work without error.
+
+
+ Command output
+
+```console title="Output"
+ N E X T F L O W ~ version 25.04.3
+
+Launching `3-main.nf` [trusting_lovelace] DSL2 - revision: 028a841db1
+
+executor > local (8)
+[ee/4ca1f2] sayHello (3) | 3 of 3 ✔
+[20/2596a7] convertToUpper (1) | 3 of 3 ✔
+[b3/e15de5] collectGreetings | 1 of 1 ✔
+[c5/af5f88] cowpy | 1 of 1 ✔
+```
+
+
+
+Behind the scenes, Nextflow has retrieved the Conda packages and created the environment, which normally takes a bit of work; so it's nice that we don't have to do any of that ourselves!
+
+!!! Tip
+
+ This runs quickly because the `cowpy` package is quite small, but if you're working with large packages, it may take a bit longer than usual the first time, and you might see the console output stay 'stuck' for a minute or so before completing.
+ This is normal and is due to the extra work Nextflow does the first time you use a new package.
+
+From our standpoint, it looks like it works exactly the same as running with Docker, even though on the backend the mechanics are a bit different.
+
+This means we're all set to run with Conda environments if needed.
+
+!!! Tip
+
+ Since these directives are assigned per process, it is possible 'mix and match', _i.e._ configure some of the processes in your workflow to run with Docker and others with Conda, for example, if the compute infrastructure you are using supports both.
+ In that case, you would enable both Docker and Conda in your configuration file.
+ If both are available for a given process, Nextflow will prioritize containers.
+
+ And as noted earlier, Nextflow supports multiple other software packaging and container technologies, so you are not limited to just those two.
+
+### Takeaway
+
+You know how to configure which software package each process should use, and how to switch between technologies.
+
+### What's next?
+
+Learn how to specify what executor Nextflow should use to actually do the work.
+
+---
+
+## 2. Specify what executor should be used to do the work
+
+Until now, we have been running our pipeline with the local executor.
+This executes each task on the machine that Nextflow is running on.
+When Nextflow begins, it looks at the available CPUs and memory.
+If the resources of the tasks ready to run exceed the available resources, Nextflow will hold the last tasks back from execution until one or more of the earlier tasks have finished, freeing up the necessary resources.
+
+For very large workloads, you may discover that your local machine is a bottleneck, either because you have a single task that requires more resources than you have available, or because you have so many tasks that waiting for a single machine to run them would take too long.
+The local executor is convenient and efficient, but is limited to that single machine.
+Nextflow supports [many different execution backends](https://www.nextflow.io/docs/latest/executor.html), including HPC schedulers (Slurm, LSF, SGE, PBS, Moab, OAR, Bridge, HTCondor and others) as well as cloud execution backends such (AWS Batch, Google Cloud Batch, Azure Batch, Kubernetes and more).
+
+### 2.1. Targeting a different backend
+
+The choice of executor is set by a process directive called `executor`.
+By default it is set to `local`, so the following configuration is implied:
+
+```groovy title="Built-in configuration"
+process {
+ executor = 'local'
+}
+```
+
+To set the executor to target a different backend, simply specify the executor you want using similar syntax as described above for resource allocations (see [documentation](https://www.nextflow.io/docs/latest/executor.html) for all options).
+
+```groovy title="nextflow.config"
+process {
+ executor = 'slurm'
+}
+```
+
+!!! Warning
+
+ We can't actually test this in the training environment because it's not set up to connect to an HPC.
+
+### 2.2. Dealing with backend-specific syntax for execution parameters
+
+Most high-performance computing platforms allow (and sometimes require) that you specify certain parameters such as resource allocation requests and limitations (for e.g. number of CPUs and memory) and name of the job queue to use.
+
+Unfortunately, each of these systems uses different technologies, syntaxes and configurations for defining how a job should be defined and submitted to the relevant scheduler.
+
+For example, a job requiring 8 CPUs and 4GB of RAM to be executed on the queue "my-science-work" needs to be expressed in the following ways depending on the backend:
+
+
+ Config for SLURM / submit using `sbatch`
+
+```bash
+#SBATCH -o /path/to/my/task/directory/my-task-1.log
+#SBATCH --no-requeue
+#SBATCH -c 8
+#SBATCH --mem 4096M
+#SBATCH -p my-science-work
+```
+
+
+
+
+ Config for PBS / submit using `qsub`
+
+```bash
+#PBS -o /path/to/my/task/directory/my-task-1.log
+#PBS -j oe
+#PBS -q my-science-work
+#PBS -l nodes=1:ppn=5
+#PBS -l mem=4gb
+```
+
+
+
+
+ Config for SGE / submit using `qsub`
+
+```bash
+#$ -o /path/to/my/task/directory/my-task-1.log
+#$ -j y
+#$ -terse
+#$ -notify
+#$ -q my-science-work
+#$ -l slots=5
+#$ -l h_rss=4096M,mem_free=4096M
+```
+
+
+
+Fortunately, Nextflow simplifies all of this.
+It provides a standardized syntax so that you can specify the relevant properties such as `cpus`, `memory` and `queue` (see documentation for other properties) just once.
+Then, at runtime, Nextflow will use those settings to generate the appropriate backend-specific scripts based on the executor setting.
+
+We'll cover that standardized syntax in the next section.
+
+### Takeaway
+
+You now know how to change the executor to use different kinds of computing infrastructure.
+
+### What's next?
+
+Learn how to evaluate and express resource allocations and limitations in Nextflow.
+
+---
+
+## 3. Allocate compute resources with process directives
+
+As noted above, high-performance computing system generally allow or require you to specify request allocations and set limitations for compute resources such as the number of CPUs and memory to use.
+
+By default, Nextflow will use a single CPU and 2GB of memory for each process.
+The corresponding process directives are called `cpus` and `memory`, so the following configuration is implied:
+
+```groovy title="Built-in configuration" linenums="1"
+process {
+ cpus = 1
+ memory = 2.GB
+}
+```
+
+You can modify these values, either for all processes or for specific named processes, using additional process directives in your configuration file.
+Nextflow will translate them into the appropriate instructions for the chosen executor.
+
+But how do you know what values to use?
+
+### 3.1. Run the workflow to generate a resource utilization report
+
+If you don't know up front how much CPU and memory your processes are likely to need, you can do some resource profiling, meaning you run the workflow with some default allocations, record how much each process used, and from there, estimate how to adjust the base allocations.
+
+Conveniently, Nextflow includes built-in tools for doing this, and will happily generate a report for you on request.
+
+To do so, add `-with-report .html` to your command line.
+
+```bash
+nextflow run 3-main.nf -with-report report-config-1.html
+```
+
+The report is an html file, which you can download and open in your browser. You can also right click it in the file explorer on the left and click on `Show preview` in order to view it in the training environment.
+
+
+
+Take a few minutes to look through the report and see if you can identify some opportunities for adjusting resources.
+Make sure to click on the tabs that show the utilization results as a percentage of what was allocated.
+There is some [documentation](https://www.nextflow.io/docs/latest/reports.html) describing all the available features.
+
+### 3.2. Set resource allocations for all processes
+
+The profiling shows that the processes in our training workflow are very lightweight, so let's reduce the default memory allocation to 1GB per process.
+
+Add the following to your `nextflow.config` file:
+
+```groovy title="nextflow.config" linenums="4"
+process {
+ memory = 1.GB
+}
+```
+
+### 3.3. Set resource allocations for an individual process
+
+At the same time, we're going to pretend that the `cowpy` process requires more resources than the others, just so we can demonstrate how to adjust allocations for an individual process.
+
+=== "After"
+
+ ```groovy title="nextflow.config" linenums="4" hl_lines="3-6"
+ process {
+ memory = 1.GB
+ withName: 'cowpy' {
+ memory = 2.GB
+ cpus = 2
+ }
+ }
+ ```
+
+=== "Before"
+
+ ```groovy title="nextflow.config" linenums="14"
+ process {
+ memory = 1.GB
+ }
+ ```
+
+With this configuration, all processes will request 1GB of memory and a single CPU (the implied default), except the `cowpy` process, which will request 2GB and 2 CPUs.
+
+!!! Tip
+
+ If you have a machine with few CPUs and you allocate a high number per process, you might see process calls getting queued behind each other.
+ This is because Nextflow ensures we don't request more CPUs than are available.
+
+You could then run the workflow again, supplying a different filename for the profiling report, and compare performance before and after the configuration changes.
+You may not notice any real difference since this is such a small workload, but this is the approach you would use to analyze the performance and resource requirements of a real-world workflow.
+
+It is very useful when your processes have different resource requirements. It empowers you to right-size the resource allocations you set up for each process based on actual data, not guesswork.
+
+!!! Tip
+
+ This is just a tiny taster of what you can do to optimize your use of resources.
+ Nextflow itself has some really neat [dynamic retry logic](https://training.nextflow.io/basic_training/debugging/#dynamic-resources-allocation) built in to retry jobs that fail due to resource limitations.
+ Additionally, the Seqera Platform offers AI-driven tooling for optimizing your resource allocations automatically.
+
+### 3.4. Add resource limits
+
+Depending on what computing executor and compute infrastructure you're using, there may be some constraints on what you can (or must) allocate.
+For example, your cluster may require you to stay within certain limits.
+
+You can use the `resourceLimits` directive to set the relevant limitations. The syntax looks like this when it's by itself in a process block:
+
+```groovy title="Syntax example"
+process {
+ resourceLimits = [
+ memory: 750.GB,
+ cpus: 200,
+ time: 30.d
+ ]
+}
+```
+
+Nextflow will translate these values into the appropriate instructions depending on the executor that you specified.
+
+As previously, we can't demonstrate this in action since we don't have access to relevant infrastructure in the training environment.
+However, if you were to set the executor to `slurm`, try running the workflow with resource allocations that exceed these limits, then look up the `sbatch` command in the `.command.run` script file (which will be generated even though the run is doomed to fail), you would see that the requests that would get sent to the executor are capped at the values specified by `resourceLimits`.
+
+!!! Tip
+
+ The nf-core project has compiled a [collection of configuration files](https://nf-co.re/configs/) shared by various institutions around the world, covering a wide range of HPC and cloud executors.
+
+ Those shared configs are valuable both for people who work there and can therefore just utilize their institution's configuration out of the box, and as a model for people who are looking to develop a configuration for their own infrastructure.
+
+### Takeaway
+
+You know how to generate a profiling report to assess resource utilization and how to modify resource allocations for all processes and/or for individual processes, as well as set resource limitations for running on HPC.
+
+### What's next?
+
+Learn how to manage workflow parameters.
+
+---
+
+## 4. Manage workflow parameters
+
+So far we've been looking at configuration from the technical point of view of the compute infrastructure.
+Now let's consider another aspect of workflow configuration that is very important for reproducibility: the configuration of the workflow parameters.
+
+Currently, our workflow is set up to accept a couple of parameter values via the command-line.
+This is fine for a simple workflow with very few parameters that need to be set for a given run.
+However, many real-world workflows will have many more parameters that may be run-specific, and putting all of them in the command line would be tedious and error-prone.
+
+### 4.1. Specify default parameter values
+
+It is possible to specify default values in the workflow script itself; for example you may see something like this in the main body of the workflow:
+
+```groovy title="Syntax example"
+params.input = 'greetings.csv'
+params.character = 'turkey'
+```
+
+The same syntax can also be used to store parameter defaults in the `nextflow.config` file.
+Let's try that out.
+
+Open the `nextflow.config` file and add the following lines to it:
+
+```groovy title="nextflow.config" linenums="1"
+/*
+ * Pipeline parameters
+ */
+params.input = 'greetings.csv'
+params.character = 'turkey'
+```
+
+
+ Code (full file)
+
+```groovy title="nextflow.config" linenums="1"
+docker.enabled = false
+conda.enabled = true
+
+process {
+ memory = 1.GB
+ withName: 'cowpy' {
+ memory = 2.GB
+ cpus = 2
+ }
+}
+
+/*
+ * Pipeline parameters
+ */
+params.input = 'greetings.csv'
+params.character = 'turkey'
+```
+
+
+
+Now you can run the workflow without specifying the parameters on the command line.
+
+```bash
+nextflow run 3-main.nf
+```
+
+This will produce the same output, but is more convenient to type, especially when the workflow requires multiple parameters.
+
+
+ Command output
+
+```console linenums="1"
+ N E X T F L O W ~ version 25.04.3
+
+Launching `3-main.nf` [wise_mahavira] DSL2 - revision: 356df0818d
+
+executor > local (8)
+[2e/d12fcb] sayHello (2) [100%] 3 of 3 ✔
+[a0/5799b6] convertToUpper (3) [100%] 3 of 3 ✔
+[db/d3bbb6] collectGreetings [100%] 1 of 1 ✔
+[a9/f75d13] cowpy [100%] 1 of 1 ✔
+```
+
+
+
+The final output file should contain the turkey character saying the greetings.
+
+
+ File contents
+
+```console title="results/cowpy-COLLECTED-output.txt"
+ _________
+/ HELLO \
+| BONJOUR |
+\ HOLà /
+ ---------
+ \ ,+*^^*+___+++_
+ \ ,*^^^^ )
+ \ _+* ^**+_
+ \ +^ _ _++*+_+++_, )
+ _+^^*+_ ( ,+*^ ^ \+_ )
+ { ) ( ,( ,_+--+--, ^) ^\
+ { (\@) } f ,( ,+-^ __*_*_ ^^\_ ^\ )
+ {:;-/ (_+*-+^^^^^+*+*<_ _++_)_ ) ) /
+ ( / ( ( ,___ ^*+_+* ) < < \
+ U _/ ) *--< ) ^\-----++__) ) ) )
+ ( ) _(^)^^)) ) )\^^^^^))^*+/ / /
+ ( / (_))_^)) ) ) ))^^^^^))^^^)__/ +^^
+ ( ,/ (^))^)) ) ) ))^^^^^^^))^^) _)
+ *+__+* (_))^) ) ) ))^^^^^^))^^^^^)____*^
+ \ \_)^)_)) ))^^^^^^^^^^))^^^^)
+ (_ ^\__^^^^^^^^^^^^))^^^^^^^)
+ ^\___ ^\__^^^^^^))^^^^^^^^)\\
+ ^^^^^\uuu/^^\uuu/^^^^\^\^\^\^\^\^\^\
+ ___) >____) >___ ^\_\_\_\_\_\_\)
+ ^^^//\\_^^//\\_^ ^(\_\_\_\)
+ ^^^ ^^ ^^^ ^
+
+```
+
+
+
+You can override those defaults by providing parameter values on the command line, or by providing them through another source of configuration information.
+
+### 4.2. Override defaults with a run-specific config file
+
+You may want to override those defaults without having to either specify parameters on the command line, or modify the original script file.
+
+A clean way to do this is to create a new `nextflow.config` file in a run-specific working directory.
+
+Let's start by creating a new directory:
+
+```bash
+mkdir -p tux-run
+cd tux-run
+```
+
+Then, create a blank configuration file in that directory:
+
+```bash
+touch nextflow.config
+```
+
+Now open the new file and add the parameters you want to customize:
+
+```groovy title="tux-run/nextflow.config" linenums="1"
+params.input = '../greetings.csv'
+params.character = 'tux'
+```
+
+Note that the path to the input file must reflect the directory structure.
+
+We can now run our pipeline from within our new working directory:
+
+```bash
+nextflow run ../3-main.nf
+```
+
+This will create a new set of directories under `tux-run/` including `tux-run/work/` and `tux-run/results/`.
+
+
+ Command output
+
+```console linenums="1"
+ N E X T F L O W ~ version 25.04.3
+
+Launching `../3-main.nf` [trusting_escher] DSL2 - revision: 356df0818d
+
+executor > local (8)
+[59/b66913] sayHello (2) [100%] 3 of 3 ✔
+[ad/f06364] convertToUpper (3) [100%] 3 of 3 ✔
+[10/714895] collectGreetings [100%] 1 of 1 ✔
+[88/3ece98] cowpy [100%] 1 of 1 ✔
+```
+
+
+
+In this run, Nextflow combines the `nextflow.config` in our current directory with the `nextflow.config` in the root directory of the pipeline, and thereby overrides the default character (turkey) with the tux character.
+
+The final output file should contain the tux character saying the greetings.
+
+
+ File contents
+
+```console title="results/cowpy-COLLECTED-output.txt"
+ _________
+/ HELLO \
+| BONJOUR |
+\ HOLà /
+ ---------
+ \
+ \
+ .--.
+ |o_o |
+ |:_/ |
+ // \ \
+ (| | )
+ /'\_ _/`\
+ \___)=(___/
+
+```
+
+
+
+That's it!
+
+Make sure to change back to the previous directory before moving to the next section.
+
+```bash
+cd ..
+```
+
+Now let's look at another useful way to set parameter values.
+
+### 4.3. Specify parameters using a parameter file
+
+Nextflow also allows us to specify parameters via a parameter file in either YAML or JSON format.
+This makes it very convenient to manage and distribute alternative sets of default values, for example, as well as run-specific parameter values.
+
+We provide an example YAML parameter file in the current directory, called `test-params.yaml`, which contains a key-value pair for each of the inputs our workflow expects.
+
+
+ File contents
+
+```yaml title="test-params.yaml" linenums="1"
+input: "greetings.csv"
+character: "stegosaurus"
+```
+
+
+
+To run the workflow with this parameter file, simply add `-params-file ` to the base command.
+
+```bash
+nextflow run 3-main.nf -params-file test-params.yaml
+```
+
+This should run without error.
+
+
+ Command output
+
+```console title="Output"
+ N E X T F L O W ~ version 25.04.3
+
+Launching `3-main.nf` [disturbed_sammet] DSL2 - revision: ede9037d02
+
+executor > local (8)
+[f0/35723c] sayHello (2) | 3 of 3 ✔
+[40/3efd1a] convertToUpper (3) | 3 of 3 ✔
+[17/e97d32] collectGreetings | 1 of 1 ✔
+[98/c6b57b] cowpy | 1 of 1 ✔
+```
+
+
+
+The final output file should contain the stegosaurus character saying the greetings.
+
+
+ File contents
+
+```console title="results/cowpy-COLLECTED-output.txt"
+_________
+/ HELLO \
+| HOLà |
+\ BONJOUR /
+ ---------
+\ . .
+ \ / `. .' "
+ \ .---. < > < > .---.
+ \ | \ \ - ~ ~ - / / |
+ _____ ..-~ ~-..-~
+ | | \~~~\.' `./~~~/
+ --------- \__/ \__/
+ .' O \ / / \ "
+ (_____, `._.' | } \/~~~/
+ `----. / } | / \__/
+ `-. | / | / `. ,~~|
+ ~-.__| /_ - ~ ^| /- _ `..-'
+ | / | / ~-. `-. _ _ _
+ |_____| |_____| ~ - . _ _ _ _ _>
+```
+
+
+
+Using a parameter file may seem like overkill when you only have a few parameters to specify, but some pipelines expect dozens of parameters.
+In those cases, using a parameter file will allow us to provide parameter values at runtime without having to type massive command lines and without modifying the workflow script.
+It also makes it easier to distribute sets of parameters to collaborators.
+
+### Takeaway
+
+You know how to manage parameter defaults and override them at runtime using command-line arguments or a parameter file.
+There are a few more options but these are the ones you are most likely to encounter.
+
+### What's next?
+
+Learn how to bring it all together by using profiles to switch between alternative configurations more conveniently.
+
+---
+
+## 5. Use profiles to select preset configurations
+
+You may want to switch between alternative settings depending on what computing infrastructure you're using. For example, you might want to develop and run small-scale tests locally on your laptop, then run full-scale workloads on HPC or cloud.
+
+This applies to workflow parameters too: you may have different sets of reference files or groups of settings that you want to swap out depending on the data you're analyzing (e.g. mouse vs human data etc).
+
+Nextflow lets you set up profiles that describe different configurations, which you can then select at runtime using a command-line argument, rather than having to modify the configuration file itself.
+
+### 5.1. Create profiles for switching between local development and execution on HPC
+
+Let's set up two alternative profiles; one for running small scale loads on a regular computer, where we'll use Docker containers, and one for running on a university HPC with a Slurm scheduler, where we'll use Conda packages.
+
+Add the following to your `nextflow.config` file:
+
+```groovy title="nextflow.config" linenums="3"
+profiles {
+ my_laptop {
+ process.executor = 'local'
+ docker.enabled = true
+ }
+ univ_hpc {
+ process.executor = 'slurm'
+ conda.enabled = true
+ process.resourceLimits = [
+ memory: 750.GB,
+ cpus: 200,
+ time: 30.d
+ ]
+ }
+}
+```
+
+You see that for the university HPC, we're also specifying resource limitations.
+
+### 5.2. Run the workflow with a profile
+
+To specify a profile in our Nextflow command line, we use the `-profile` argument.
+
+Let's try running the workflow with the `my_laptop` configuration.
+
+```bash
+nextflow run 3-main.nf -profile my_laptop
+```
+
+This should run without error and produce the same results as previously.
+
+
+ Command output
+
+```console
+ N E X T F L O W ~ version 25.04.3
+
+Launching `3-main.nf` [gigantic_brazil] DSL2 - revision: ede9037d02
+
+executor > local (8)
+[58/da9437] sayHello (3) | 3 of 3 ✔
+[35/9cbe77] convertToUpper (2) | 3 of 3 ✔
+[67/857d05] collectGreetings | 1 of 1 ✔
+[37/7b51b5] cowpy | 1 of 1 ✔
+```
+
+
+
+As you can see, this allows us to toggle between configurations very conveniently at runtime.
+
+!!! Warning
+
+ The `univ_hpc` profile will not run properly in the training environment since we do not have access to a Slurm scheduler.
+
+If in the future we find other elements of configuration that are always co-occurring with these, we can simply add them to the corresponding profile(s).
+We can also create additional profiles if there are other elements of configuration that we want to group together.
+
+### 5.3. Create a test profile
+
+As noted above, profiles are not only for infrastructure configuration.
+We can also use them to swap out sets of default values for workflow parameters, or to make it easier for ourselves and for others to try out the workflow without having to gather appropriate input values themselves.
+
+Let's take the example of creating a test profile to make it easy to test the workflow with minimal effort.
+
+The syntax for expressing default values is the same as when writing them into the workflow file itself, except we wrap them in a block named `test`:
+
+```groovy title="Syntax example"
+ test {
+ params.
+ params.
+ ...
+ }
+```
+
+If we add a test profile for our workflow, the `profiles` block becomes:
+
+```groovy title="nextflow.config" linenums="4"
+profiles {
+ my_laptop {
+ process.executor = 'local'
+ docker.enabled = true
+ }
+ univ_hpc {
+ process.executor = 'slurm'
+ conda.enabled = true
+ process.resourceLimits = [
+ memory: 750.GB,
+ cpus: 200,
+ time: 30.d
+ ]
+ }
+ test {
+ params.input = 'greetings.csv'
+ params.character = 'turtle'
+ }
+}
+```
+
+Just like for technical configuration profiles, you can set up multiple different profiles specifying workflow parameters under any arbitrary name you like.
+
+### 5.4. Run the workflow locally with the test profile
+
+Conveniently, profiles are not mutually exclusive, so we can specify multiple profiles in our command line using the following syntax `-profile ,` (for any number of profiles).
+
+!!! Tip
+
+ If you combine profiles that set values for the same elements of configuration and are described in the same configuration file, Nextflow will resolve the conflict by using whichever value it read in last (_i.e._ whatever comes later in the file).
+
+Let's try adding the test profile to our previous command:
+
+```bash
+nextflow run 3-main.nf -profile my_laptop,test
+```
+
+This should run without error.
+
+
+ Command output
+
+```console
+ N E X T F L O W ~ version 25.04.3
+
+Launching `3-main.nf` [gigantic_brazil] DSL2 - revision: ede9037d02
+
+executor > local (8)
+[58/da9437] sayHello (3) | 3 of 3 ✔
+[35/9cbe77] convertToUpper (2) | 3 of 3 ✔
+[67/857d05] collectGreetings | 1 of 1 ✔
+[37/7b51b5] cowpy | 1 of 1 ✔
+```
+
+
+
+The final output file should contain the turtle character saying the greetings.
+
+
+ File contents
+
+```console title="results/cowpy-COLLECTED-output.txt"
+ _________
+/ BONJOUR \
+| HOLà |
+\ HELLO /
+ ---------
+ \ ___-------___
+ \ _-~~ ~~-_
+ \ _-~ /~-_
+ /^\__/^\ /~ \ / \
+ /| O|| O| / \_______________/ \
+ | |___||__| / / \ \
+ | \ / / \ \
+ | (_______) /______/ \_________ \
+ | / / \ / \
+ \ \^\\ \ / \ /
+ \ || \______________/ _-_ //\__//
+ \ ||------_-~~-_ ------------- \ --/~ ~\ || __/
+ ~-----||====/~ |==================| |/~~~~~
+ (_(__/ ./ / \_\ \.
+ (_(___/ \_____)_)
+```
+
+
+
+This means that as long as we distribute any test data files with the workflow code, anyone can quickly try out the workflow without having to supply their own inputs via the command line or a parameter file.
+
+!!! Tip
+
+ You can even point to URLs for larger files that are stored externally.
+ Nextflow will download them automatically as long as there is an open connection.
+
+### 5.5. Use `nextflow config` to see the resolved configuration
+
+As noted above, sometimes the same parameter can be set to different values in profiles that you want to combine.
+And more generally, there are numerous places where elements of configuration can be stored, and sometimes the same properties can be set to different values in different places.
+
+Nextflow applies a set [order of precedence](https://www.nextflow.io/docs/latest/config.html) to resolve any conflicts, but that can be tricky to determine yourself.
+And even if nothing is conflicting, it can be tedious to look up all the possible places where things could be configured.
+
+Fortunately, Nextflow includes a convenient utility tool called `config` that can automate that whole process for you.
+
+The `config` tool will explore all the contents in your current working directory, hoover up any configuration files, and produce the fully resolved configuration that Nextflow would use to run the workflow.
+This allows you to find out what settings will be used without having to launch anything.
+
+#### 5.5.1. Resolve the default configuration
+
+Run this command to resolve the configuration that would be applied by default.
+
+```bash
+nextflow config
+```
+
+
+ Command output
+
+```groovy
+docker {
+ enabled = false
+}
+
+conda {
+ enabled = true
+}
+
+process {
+ memory = '1 GB'
+ withName:cowpy {
+ memory = '2 GB'
+ cpus = 2
+ }
+}
+
+params {
+ input = 'greetings.csv'
+ character = 'turkey'
+}
+```
+
+
+
+#### 5.5.2. Resolve the configuration with specific settings activated
+
+If you provide command-line parameters, e.g. enabling one or more profiles or loading a parameter file, the command will additionally take those into account.
+
+```bash
+nextflow config -profile my_laptop,test
+```
+
+
+ Command output
+
+```groovy
+docker {
+ enabled = true
+}
+
+conda {
+ enabled = true
+}
+
+process {
+ memory = '1 GB'
+ withName:cowpy {
+ memory = '2 GB'
+ cpus = 2
+ }
+ executor = 'local'
+}
+
+params {
+ input = 'greetings.csv'
+ character = 'turtle'
+}
+```
+
+
+
+This gets especially useful for complex projects that involve multiple layers of configuration.
+
+### Takeaway
+
+You know how to use profiles to select a preset configuration at runtime with minimal hassle.
+More generally, you know how to configure your workflow executions to suit different compute platforms and enhance the reproducibility of your analyses.
+
+### What's next?
+
+Give yourself a big pat on the back!
+You know everything you need to know to get started running and managing Nextflow pipelines.
+
+That concludes this course, but if you're eager to keep learning, we have two main recommendations:
+
+- If you want to dig deeper into developing your own pipelines, have a look at [Hello Nextflow](../hello_nextflow/index.md), a course for beginners that covers the same general progression as this one but goes into much more detail about channels and operators.
+- If you would like to continue learning how to run Nextflow pipelines without going deeper into the code, have a look at the first part of [Hello nf-core](../hello_nf-core/index.md), which introduces the tooling for finding and running pipelines from the hugely popular [nf-core](https://nf-co.re/) project.
+
+Have fun!
diff --git a/docs/nextflow_run/03_run_nf-core.md b/docs/nextflow_run/03_run_nf-core.md
deleted file mode 100644
index 0b78d3b73..000000000
--- a/docs/nextflow_run/03_run_nf-core.md
+++ /dev/null
@@ -1,1380 +0,0 @@
-# Part 3: Run nf-core
-
-nf-core is a community effort to develop and maintain a curated set of analysis pipelines built using Nextflow.
-
-
-
-nf-core provides a standardized set of best practices, guidelines, and templates for building and sharing scientific pipelines.
-These pipelines are designed to be modular, scalable, and portable, allowing researchers to easily adapt and execute them using their own data and compute resources.
-
-One of the key benefits of nf-core is that it promotes open development, testing, and peer review, ensuring that the pipelines are robust, well-documented, and validated against real-world datasets.
-This helps to increase the reliability and reproducibility of scientific analyses and ultimately enables researchers to accelerate their scientific discoveries.
-
-nf-core is published in Nature Biotechnology: [Nat Biotechnol 38, 276–278 (2020). Nature Biotechnology](https://www.nature.com/articles/s41587-020-0439-x). An updated preprint is available at [bioRxiv](https://www.biorxiv.org/content/10.1101/2024.05.10.592912v1).
-
-## nf-core pipelines and other components
-
-The nf-core collection currently offers [over 100 pipelines](https://nf-co.re/pipelines/) in various stages of development, [72 subworkflows](https://nf-co.re/subworkflows/) and [over 1300 modules](https://nf-co.re/modules/) that you can use to build your own pipelines.
-
-Each released pipeline has a dedicated page that includes 6 documentation sections:
-
-- **Introduction:** An introduction and overview of the pipeline
-- **Usage:** Descriptions of how to execute the pipeline
-- **Parameters:** Grouped pipeline parameters with descriptions
-- **Output:** Descriptions and examples of the expected output files
-- **Results:** Example output files generated from the full test dataset
-- **Releases & Statistics:** Pipeline version history and statistics
-
-You should read the pipeline documentation carefully to understand what a given pipeline does and how it can be configured before attempting to run it.
-
-### Pulling an nf-core pipeline
-
-One really cool aspect of how Nextflow manages pipelines is that you can pull a pipeline from a GitHub repository without cloning the repository.
-This is really convenient if you just want to run a pipeline without modifying the code.
-
-So if you want to try out an nf-core pipeline with minimal effort, you can start by pulling it using the `nextflow pull` command.
-
-!!!tip
-
- You can run this from anywhere, but if you feel like being consistent with previous exercises, you can create a `nf-core-demo` directory under `hello-nextflow`. If you were working through Part 7 (Hello nf-test) before this, you may need to go up one level first.
-
- ```bash
- mkdir nf-core-demo
- cd nf-core-demo
- ```
-
-Whenever you're ready, run the command:
-
-```bash
-nextflow pull nf-core/demo
-```
-
-Nextflow will `pull` the pipeline's default GitHub branch.
-For nf-core pipelines with a stable release, that will be the master branch.
-You select a specific branch with `-r`; we'll cover that later.
-
-```console title="Output"
-Checking nf-core/demo ...
- downloaded from https://github.com/nf-core/demo.git - revision: 04060b4644 [master]
-```
-
-To be clear, you can do this with any Nextflow pipeline that is appropriately set up in GitHub, not just nf-core pipelines.
-However nf-core is the largest open curated collection of Nextflow pipelines.
-
-!!!tip
-
- Pulled pipelines are stored in a hidden assets folder. By default, this folder is `$HOME/.nextflow/assets`, but in this training environment the folder has been set to `$NXF_HOME/assets`:
-
- ```bash
- tree $NXF_HOME/assets/ -L 2
- ```
-
- ```console title="Output"
- /home/gitpod/.nextflow/assets/
- └── nf-core
- └── demo
- ```
-
- So you don't actually see them listed in your working directory.
- However, you can view a list of your cached pipelines using the `nextflow list` command:
-
- ```bash
- nextflow list
- ```
-
- ```console title="Output"
- nf-core/demo
- ```
-
-Now that we've got the pipeline pulled, we can try running it!
-
-### Trying out an nf-core pipeline with the test profile
-
-Conveniently, every nf-core pipeline comes with a `test` profile.
-This is a minimal set of configuration settings for the pipeline to run using a small test dataset that is hosted on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository. It's a great way to try out a pipeline at small scale.
-
-The `test` profile for `nf-core/demo` is shown below:
-
-```groovy title="conf/test.config" linenums="1"
-/*
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Nextflow config file for running minimal tests
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Defines input files and everything required to run a fast and simple pipeline test.
-
- Use as follows:
- nextflow run nf-core/demo -profile test, --outdir
-
-----------------------------------------------------------------------------------------
-*/
-
-process {
- resourceLimits = [
- cpus: 4,
- memory: '15.GB',
- time: '1.h'
- ]
-}
-
-params {
- config_profile_name = 'Test profile'
- config_profile_description = 'Minimal test dataset to check pipeline function'
-
- // Input data
- input = 'https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/samplesheet/samplesheet_test_illumina_amplicon.csv'
-
-}
-```
-
-This tells us that the `nf-core/demo` `test` profile already specifies the input parameter, so you don't have to provide any input yourself.
-However, the `outdir` parameter is not included in the `test` profile, so you have to add it to the execution command using the `--outdir` flag.
-
-Here, we're also going to specify `-profile docker`, which by nf-core convention enables the use of Docker.
-
-Lets' try it!
-
-```bash
-nextflow run nf-core/demo -profile docker,test --outdir results
-```
-
-!!! hint "Changing Nextflow version"
-
- Depending on the Nextflow version you have installed, this command might fail due to a version mismatch.
- If that happens, you can temporarily run the pipeline with a different version than you have installed by adding `NXF_VER=version` to the start of your command as shown below:
-
- ```bash
- NXF_VER=24.09.2-edge nextflow run nf-core/demo -profile docker,test --outdir results
- ```
-
-Here's the console output from the pipeline:
-
-```console title="Output"
- N E X T F L O W ~ version 24.09.2-edge
-
-Launching `https://github.com/nf-core/demo` [naughty_bell] DSL2 - revision: 04060b4644 [master]
-
-
-------------------------------------------------------
- ,--./,-.
- ___ __ __ __ ___ /,-._.--~'
- |\ | |__ __ / ` / \ |__) |__ } {
- | \| | \__, \__/ | \ |___ \`-._,-`-,
- `._,._,'
- nf-core/demo 1.0.1
-------------------------------------------------------
-Input/output options
- input : https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/samplesheet/samplesheet_test_illumina_amplicon.csv
- outdir : results
-
-Institutional config options
- config_profile_name : Test profile
- config_profile_description: Minimal test dataset to check pipeline function
-
-Core Nextflow options
- revision : master
- runName : naughty_bell
- containerEngine : docker
- launchDir : /workspaces/training/hello-nextflow
- workDir : /workspaces/training/hello-nextflow/work
- projectDir : /home/gitpod/.nextflow/assets/nf-core/demo
- userName : gitpod
- profile : docker,test
- configFiles :
-
-!! Only displaying parameters that differ from the pipeline defaults !!
-------------------------------------------------------* The pipeline
- https://doi.org/10.5281/zenodo.12192442
-
-* The nf-core framework
- https://doi.org/10.1038/s41587-020-0439-x
-
-* Software dependencies
- https://github.com/nf-core/demo/blob/master/CITATIONS.md
-
-executor > local (7)
-[0a/e694d8] NFCORE_DEMO:DEMO:FASTQC (SAMPLE3_SE) [100%] 3 of 3 ✔
-[85/4198c1] NFCORE_DEMO:DEMO:SEQTK_TRIM (SAMPLE1_PE) [100%] 3 of 3 ✔
-[d8/fe153e] NFCORE_DEMO:DEMO:MULTIQC [100%] 1 of 1 ✔
--[nf-core/demo] Pipeline completed successfully-
-Completed at: 28-Oct-2024 03:24:58
-Duration : 1m 13s
-CPU hours : (a few seconds)
-Succeeded : 7
-```
-
-Isn't that neat?
-
-You can also explore the `results` directory produced by the pipeline.
-
-```console title="Output"
-results
-├── fastqc
-│ ├── SAMPLE1_PE
-│ ├── SAMPLE2_PE
-│ └── SAMPLE3_SE
-├── fq
-│ ├── SAMPLE1_PE
-│ ├── SAMPLE2_PE
-│ └── SAMPLE3_SE
-├── multiqc
-│ ├── multiqc_data
-│ ├── multiqc_plots
-│ └── multiqc_report.html
-└── pipeline_info
- ├── execution_report_2024-10-28_03-23-44.html
- ├── execution_timeline_2024-10-28_03-23-44.html
- ├── execution_trace_2024-10-28_03-14-32.txt
- ├── execution_trace_2024-10-28_03-19-33.txt
- ├── execution_trace_2024-10-28_03-20-57.txt
- ├── execution_trace_2024-10-28_03-22-39.txt
- ├── execution_trace_2024-10-28_03-23-44.txt
- ├── nf_core_pipeline_software_mqc_versions.yml
- ├── params_2024-10-28_03-23-49.json
- └── pipeline_dag_2024-10-28_03-23-44.html
-```
-
-If you're curious about what that all means, check out [the nf-core/demo pipeline documentation page](https://nf-co.re/demo/1.0.1/)!
-
-And that's all you need to know for now.
-Congratulations! You have now run your first nf-core pipeline.
-
-### Takeaway
-
-You have a general idea of what nf-core offers and you know how to run an nf-core pipeline using its built-in test profile.
-
-### What's next?
-
-Celebrate and take another break! Next, we'll show you how to use nf-core tooling to build your own pipeline.
-
-## Create a basic pipeline from template
-
-We will now start developing our own nf-core style pipeline. The nf-core community provides a [command line tool](https://nf-co.re/docs/nf-core-tools) with helper functions to use and develop pipelines.
-We have pre-installed nf-core tools, and here, we will use them to create and develop a new pipeline.
-
-View all of the tooling using the `nf-core --help` argument.
-
-```bash
-nf-core --help
-```
-
-### Creating your pipeline
-
-Before we start, let's navigate into the `hello-nf-core` directory:
-
-```
-cd ..
-cd hello-nf-core
-```
-
-!!! hint "Open a new window in VSCode"
-
- If you are working with VS Code you can open a new window to reduce visual clutter:
-
- ```bash
- code .
- ```
-
-Let's start by creating a new pipeline with the `nf-core pipelines create` command:
-
-All nf-core pipelines are based on a common template, a standardized pipeline skeleton that can be used to streamline development with shared features and components.
-
-The `nf-core pipelines create` command creates a new pipeline using the nf-core base template with a pipeline name, description, and author. It is the first and most important step for creating a pipeline that will integrate with the wider Nextflow ecosystem.
-
-```bash
-nf-core pipelines create
-```
-
-Running this command will open a Text User Interface (TUI) for pipeline creation.
-
-
-
-
-
-Template features can be flexibly included or excluded at the time of creation, follow these steps create your first pipeline using the `nf-core pipelines create` TUI:
-
-1. Run the `nf-core pipelines create` command
-2. Select **Let's go!** on the welcome screen
-3. Select **Custom** on the Choose pipeline type screen
-4. Enter your pipeline details, replacing < YOUR NAME > with your own name, then select **Next**
-
- - **GitHub organisation:** myorg
- - **Workflow name:** myfirstpipeline
- - **A short description of your pipeline:** My first pipeline
- - **Name of the main author / authors:** < YOUR NAME >
-
-5. On the Template features screen, turn **off**:
-
- - `Use a GitHub repository`
- - `Add GitHub CI tests`
- - `Use reference genomes`
- - `Add GitHub badges`
- - `Include citations`
- - `Include a gitpod environment`
- - `Include GitHub Codespaces`
- - `Use fastqc`
- - `Add a changelog`
- - `Support Microsoft Teams notifications`
- - `Support Slack notifications`
-
-6. Select **Finish** on the Final details screen
-7. Wait for the pipeline to be created, then select **Continue**
-8. Select **Finish without creating a repo** on the Create GitHub repository screen
-9. Select **Close** on the HowTo create a GitHub repository page
-
-If run successfully, you will see a new folder in your current directory named `myorg-myfirstpipeline`.
-
-### Testing your pipeline
-
-Let's try to run our new pipeline:
-
-```bash
-cd myorg-myfirstpipeline
-nextflow run . -profile docker,test --outdir results
-```
-
-The pipeline should run successfully!
-
-Here's the console output from the pipeline:
-
-```console title="Output"
-Launching `./main.nf` [marvelous_saha] DSL2 - revision: a633aedb88
-
-Input/output options
- input : https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/samplesheet/samplesheet_test_illumina_amplicon.csv
- outdir : results
-
-Institutional config options
- config_profile_name : Test profile
- config_profile_description: Minimal test dataset to check pipeline function
-
-Core Nextflow options
- runName : marvelous_saha
- containerEngine : docker
- launchDir : /workspaces/training/hello-nextflow/hello-nf-core/myorg-myfirstpipeline
- workDir : /workspaces/training/hello-nextflow/hello-nf-core/myorg-myfirstpipeline/work
- projectDir : /workspaces/training/hello-nextflow/hello-nf-core/myorg-myfirstpipeline
- userName : gitpod
- profile : docker,test
- configFiles :
-
-!! Only displaying parameters that differ from the pipeline defaults !!
-------------------------------------------------------
-executor > local (1)
-[ba/579181] process > MYORG_MYFIRSTPIPELINE:MYFIRSTPIPELINE:MULTIQC [100%] 1 of 1 ✔
--[myorg/myfirstpipeline] Pipeline completed successfully-
-```
-
-Let's dissect what we are seeing.
-
-The nf-core pipeline template is a working pipeline and comes preconfigured with some modules. Here, we only run [MultiQC](https://multiqc.info/)
-
-At the top, you see all parameters displayed that differ from the pipeline defaults. Most of these are default or were set by applying the `test` profile.
-
-Additionally we used the `docker` profile to use docker for software packaging. nf-core provides this as a profile for convenience to enable the docker feature but we could do it with configuration as we did with the earlier module.
-
-### Template tour
-
-The nf-core pipeline template comes packed with a lot of files and folders. While creating the pipeline, we selected a subset of the nf-core features. The features we selected are now included as files and directories in our repository.
-
-While the template may feel overwhelming, a complete understanding isn't required to start developing your pipeline. Let's look at the important places that we need to touch during pipeline development.
-
-#### Workflows, subworkflows, and modules
-
-The nf-core pipeline template has a `main.nf` script that calls `myfirstpipeline.nf` from the `workflows` folder. The `myfirstpipeline.nf` file inside the workflows folder is the central pipeline file that is used to bring everything else together.
-
-Instead of having one large monolithic pipeline script, it's broken up into smaller script components, namely, modules and subworkflows:
-
-- **Modules:** Wrappers around a single process
-- **Subworkflows:** Two or more modules that are packaged together as a mini workflow
-
-
- --8<-- "docs/hello_nextflow/img/nested.excalidraw.svg"
-
-
-Within your pipeline repository, `modules` and `subworkflows` are stored within `local` and `nf-core` folders. The `nf-core` folder is for components that have come from the nf-core GitHub repository while the `local` folder is for components that have been developed independently (usually things very specific to a pipeline):
-
-```console
-modules/
-├── local
-│ └── .nf
-│ .
-│
-└── nf-core
- ├──
- │ ├── environment.yml
- │ ├── main.nf
- │ ├── meta.yml
- │ └── tests
- │ ├── main.nf.test
- │ ├── main.nf.test.snap
- │ └── tags.yml
- .
-```
-
-Modules from nf-core follow a similar structure and contain a small number of additional files for testing using [nf-test](https://www.nf-test.com/) and documentation about the module.
-
-!!!note
-
- Some nf-core modules are also split into command specific directories:
-
- ```console
- │
- └──
- └──
- ├── environment.yml
- ├── main.nf
- ├── meta.yml
- └── tests
- ├── main.nf.test
- ├── main.nf.test.snap
- └── tags.yml
- ```
-
-!!!note
-
- The nf-core template does not come with a local modules folder by default.
-
-#### Configuration files
-
-The nf-core pipeline template utilizes Nextflow's flexible customization options and has a series of configuration files throughout the template.
-
-In the template, the `nextflow.config` file is a central configuration file and is used to set default values for parameters and other configuration options. The majority of these configuration options are applied by default while others (e.g., software dependency profiles) are included as optional profiles.
-
-There are several configuration files that are stored in the `conf` folder and are added to the configuration by default or optionally as profiles:
-
-- `base.config`: A 'blank slate' config file, appropriate for general use on most high-performance computing environments. This defines broad bins of resource usage, for example, which are convenient to apply to modules.
-- `modules.config`: Additional module directives and arguments.
-- `test.config`: A profile to run the pipeline with minimal test data.
-- `test_full.config`: A profile to run the pipeline with a full-sized test dataset.
-
-#### `nextflow_schema.json`
-
-The `nextflow_schema.json` is a file used to store parameter related information including type, description and help text in a machine readable format. The schema is used for various purposes, including automated parameter validation, help text generation, and interactive parameter form rendering in UI interfaces.
-
-#### `assets/schema_input.json`
-
-The `schema_input.json` is a file used to define the input samplesheet structure. Each column can have a type, pattern, description and help text in a machine readable format. The schema is used for various purposes, including automated validation, and providing helpful error messages.
-
-### Takeaway
-
-You have an example pipeline, and learned about important template files.
-
-### What's next?
-
-Congratulations! In the next step, we will check the input data.
-
----
-
-## Check the input data
-
-Above, we said that the `test` profile comes with small test files that are stored in the nf-core. Let's check what type of files we are dealing with to plan our expansion. Remember that we can inspect any channel content using the `view` operator:
-
-```groovy title="workflows/myfirstpipeline.nf" linenums="27"
-ch_samplesheet.view()
-```
-
-and the run command:
-
-```bash
-nextflow run . -profile docker,test --outdir results
-```
-
-The output should look like the below. We see that we have FASTQ files as input and each set of files is accompanied by some metadata: the `id` and whether or not they are single end:
-
-```console title="Output"
-[['id':'SAMPLE1_PE', 'single_end':false], [/nf-core/test-datasets/viralrecon/illumina/amplicon/sample1_R1.fastq.gz, /nf-core/test-datasets/viralrecon/illumina/amplicon/sample1_R2.fastq.gz]]
-[['id':'SAMPLE2_PE', 'single_end':false], [/nf-core/test-datasets/viralrecon/illumina/amplicon/sample2_R1.fastq.gz, /nf-core/test-datasets/viralrecon/illumina/amplicon/sample2_R2.fastq.gz]]
-[['id':'SAMPLE3_SE', 'single_end':true], [/nf-core/test-datasets/viralrecon/illumina/amplicon/sample1_R1.fastq.gz, /nf-core/test-datasets/viralrecon/illumina/amplicon/sample2_R1.fastq.gz]]
-```
-
-You can comment the `view` statement for now. We will use later during this training to inspect the channel content again.
-
-### Takeaway
-
-You have learned how input data is supplied via a samplesheet.
-
-### What's next?
-
-In the next step we will start changing the code and add new tools to the pipeline.
-
----
-
-## Add an nf-core module
-
-nf-core provides a large library of modules and subworkflows: pre-made nextflow wrappers around tools that can be installed into nextflow pipelines. They are designed to be flexible but may require additional configuration to suit different use cases.
-
-Currently, there are more than [1300 nf-core modules](https://nf-co.re/modules) and [60 nf-core subworkflows](https://nf-co.re/subworkflows) (November 2024) available. Modules and subworkflows can be listed, installed, updated, removed, and patched using nf-core tooling.
-
-While you could develop a module for this tool independently, you can save a lot of time and effort by leveraging nf-core modules and subworkflows.
-
-Let's see which modules are available:
-
-```console
-nf-core modules list remote
-```
-
-This command lists all currently available modules, > 1300. An easier way to find them is to go to the nf-core website and visit the modules subpage [https://nf-co.re/modules](https://nf-co.re/modules). Here you can search for modules by name or tags, find documentation for each module, and see which nf-core pipeline are using the module:
-
-
-
-### Install an nf-core module
-
-Now let's add another tool to the pipeline.
-
-`Seqtk` is a fast and lightweight tool for processing sequences in the FASTA or FASTQ format. Here, you will use the [`seqtk trim`](https://github.com/lh3/seqtk) command to trim FASTQ files.
-
-In your pipeline, you will add a new step that will take FASTQ files from the sample sheet as inputs and will produce trimmed fastq files that can be used as an input for other tools and version information about the seqtk tools to mix into the inputs for the MultiQC process.
-
-
- --8<-- "docs/hello_nextflow/img/pipeline.excalidraw.svg"
-
-
-The `nf-core modules install` command can be used to install the `seqtk/trim` module directly from the nf-core repository:
-
-```
-nf-core modules install
-```
-
-!!!warning
-
- You need to be in the myorg-myfirstpipeline directory when executing `nf-core modules install`
-
-You can follow the prompts to find and install the module you are interested in:
-
-```console
-? Tool name: seqtk/trim
-```
-
-Once selected, the tooling will install the module in the `modules/nf-core/` folder and suggest code that you can add to your main workflow file (`workflows/myfirstpipeline.nf`).
-
-```console
-INFO Installing 'seqtk/trim'
-INFO Use the following statement to include this module:
-
-include { SEQTK_TRIM } from '../modules/nf-core/seqtk/trim/main'
-```
-
-To enable reporting and reproducibility, modules and subworkflows from the nf-core repository are tracked using hashes in the `modules.json` file. When modules are installed or removed using the nf-core tooling the `modules.json` file will be automatically updated.
-
-When you open the `modules.json`, you will see an entry for each module that is currently installed from the nf-core modules repository. You can open the file with the VS Code user interface by clicking on it in `myorg-myfirstpipeline/modules.json`:
-
-```console
-"nf-core": {
- "multiqc": {
- "branch": "master",
- "git_sha": "cf17ca47590cc578dfb47db1c2a44ef86f89976d",
- "installed_by": ["modules"]
- },
- "seqtk/trim": {
- "branch": "master",
- "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
- "installed_by": ["modules"]
- }
-}
-```
-
-### Add the module to your pipeline
-
-Although the module has been installed in your local pipeline repository, it is not yet added to your pipeline.
-
-The suggested `include` statement needs to be added to your `workflows/myfirstpipeline.nf` file and the process call (with inputs) needs to be added to the workflow block.
-
-```groovy title="workflows/myfirstpipeline.nf" linenums="6"
-include { SEQTK_TRIM } from '../modules/nf-core/seqtk/trim/main'
-include { MULTIQC } from '../modules/nf-core/multiqc/main'
-```
-
-To add the `SEQTK_TRIM` module to your workflow you will need to check what inputs are required.
-
-You can view the input channels for the module by opening the `./modules/nf-core/seqtk/trim/main.nf` file.
-
-```groovy title="modules/nf-core/seqtk/trim/main.nf" linenums="11"
-input:
-tuple val(meta), path(reads)
-```
-
-Each nf-core module also has a `meta.yml` file which describes the inputs and outputs. This meta file is rendered on the [nf-core website](https://nf-co.re/modules/seqtk_trim), or can be viewed using the `nf-core modules info` command:
-
-```console
-nf-core modules info seqtk/trim
-```
-
-It outputs a table with all defined inputs and outputs of the module:
-
-```console title="Output"
-
-╭─ Module: seqtk/trim ─────────────────────────────────────────────────────────────────────────────╮
-│ Location: modules/nf-core/seqtk/trim │
-│ 🔧 Tools: seqtk │
-│ 📖 Description: Trim low quality bases from FastQ files │
-╰───────────────────────────────────────────────────────────────────────────────────────────────────╯
- ╷ ╷
- 📥 Inputs │Description │ Pattern
-╺━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━╸
- input[0] │ │
-╶──────────────┼───────────────────────────────────────────────────────────────────────┼────────────╴
- meta (map) │Groovy Map containing sample information e.g. [ id:'test', │
- │single_end:false ] │
-╶──────────────┼───────────────────────────────────────────────────────────────────────┼────────────╴
- reads (file)│List of input FastQ files │*.{fastq.gz}
- ╵ ╵
- ╷ ╷
- 📥 Outputs │Description │ Pattern
-╺━━━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━╸
- reads │ │
-╶─────────────────────┼────────────────────────────────────────────────────────────────┼────────────╴
- meta (map) │Groovy Map containing sample information e.g. [ id:'test', │
- │single_end:false ] │
-╶─────────────────────┼────────────────────────────────────────────────────────────────┼────────────╴
- *.fastq.gz (file) │Filtered FastQ files │*.{fastq.gz}
-╶─────────────────────┼────────────────────────────────────────────────────────────────┼────────────╴
- versions │ │
-╶─────────────────────┼────────────────────────────────────────────────────────────────┼────────────╴
- versions.yml (file)│File containing software versions │versions.yml
- ╵ ╵
-
- Use the following statement to include this module:
-
- include { SEQTK_TRIM } from '../modules/nf-core/seqtk/trim/main'
-```
-
-Using this module information you can work out what inputs are required for the `SEQTK_TRIM` process:
-
-1. `tuple val(meta), path(reads)`
-
- - A tuple with a meta _map_ and a list of FASTQ _files_
- - The channel `ch_samplesheet` used by the `FASTQC` process can be used as the reads input.
-
-Only one input channel is required, and it already exists, so it can be added to your `firstpipeline.nf` file without any additional channel creation or modifications.
-
-_Before:_
-
-```groovy title="workflows/myfirstpipeline.nf" linenums="30"
-//
-// Collate and save software versions
-//
-```
-
-_After:_
-
-```groovy title="workflows/myfirstpipeline.nf" linenums="29"
-//
-// MODULE: Run SEQTK_TRIM
-//
-SEQTK_TRIM (
- ch_samplesheet
-)
-//
-// Collate and save software versions
-//
-```
-
-Let's test it:
-
-```bash
-nextflow run . -profile docker,test --outdir results
-```
-
-```console title="Output"
-Launching `./main.nf` [drunk_waddington] DSL2 - revision: a633aedb88
-
-Input/output options
- input : https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/samplesheet/samplesheet_test_illumina_amplicon.csv
- outdir : results
-
-Institutional config options
- config_profile_name : Test profile
- config_profile_description: Minimal test dataset to check pipeline function
-
-Core Nextflow options
- runName : drunk_waddington
- containerEngine : docker
- launchDir : /workspaces/training/hello-nextflow/hello-nf-core/myorg-myfirstpipeline
- workDir : /workspaces/training/hello-nextflow/hello-nf-core/myorg-myfirstpipeline/work
- projectDir : /workspaces/training/hello-nextflow/hello-nf-core/myorg-myfirstpipeline
- userName : gitpod
- profile : docker,test
- configFiles :
-
-!! Only displaying parameters that differ from the pipeline defaults !!
-------------------------------------------------------
-executor > local (4)
-[74/9b2e7b] process > MYORG_MYFIRSTPIPELINE:MYFIRSTPIPELINE:SEQTK_TRIM (SAMPLE2_PE) [100%] 3 of 3 ✔
-[ea/5ca001] process > MYORG_MYFIRSTPIPELINE:MYFIRSTPIPELINE:MULTIQC [100%] 1 of 1 ✔
--[myorg/myfirstpipeline] Pipeline completed successfully-
-```
-
-### Inspect results folder
-
-Default nf-core configuration directs the output of each process into the `/`. After running the previous command, you
-should have a `results` folder that looks something like this:
-
-```console
-results
-├── multiqc
-│ ├── multiqc_data
-│ └── multiqc_report.html
-├── pipeline_info
-│ ├── execution_report_2024-11-14_12-07-43.html
-│ ├── execution_report_2024-11-14_12-12-42.html
-│ ├── execution_report_2024-11-14_12-13-58.html
-│ ├── execution_report_2024-11-14_12-28-59.html
-│ ├── execution_timeline_2024-11-14_12-07-43.html
-│ ├── execution_timeline_2024-11-14_12-12-42.html
-│ ├── execution_timeline_2024-11-14_12-13-58.html
-│ ├── execution_timeline_2024-11-14_12-28-59.html
-│ ├── execution_trace_2024-11-14_12-07-43.txt
-│ ├── execution_trace_2024-11-14_12-12-42.txt
-│ ├── execution_trace_2024-11-14_12-13-58.txt
-│ ├── execution_trace_2024-11-14_12-28-59.txt
-│ ├── params_2024-11-14_12-07-44.json
-│ ├── params_2024-11-14_12-12-43.json
-│ ├── params_2024-11-14_12-13-59.json
-│ ├── params_2024-11-14_12-29-00.json
-│ ├── pipeline_dag_2024-11-14_12-07-43.html
-│ ├── pipeline_dag_2024-11-14_12-12-42.html
-│ ├── pipeline_dag_2024-11-14_12-13-58.html
-│ ├── pipeline_dag_2024-11-14_12-28-59.html
-│ └── pipeline_software_mqc_versions.yml
-└── seqtk
- ├── SAMPLE1_PE_sample1_R1.fastq.gz
- ├── SAMPLE1_PE_sample1_R2.fastq.gz
- ├── SAMPLE2_PE_sample2_R1.fastq.gz
- ├── SAMPLE2_PE_sample2_R2.fastq.gz
- ├── SAMPLE3_SE_sample1_R1.fastq.gz
- └── SAMPLE3_SE_sample2_R1.fastq.gz
-```
-
-The outputs from the `multiqc` and `seqtk` modules are published in their respective subdirectories. In addition, by default,`nf-core' pipelines generate a set of reports. These files are stored in the`pipeline_info` subdirectory and time-stamped so that runs don't overwrite each other.
-
-### Handle modules output
-
-As with the inputs, you can view the outputs for the module by opening the `/modules/nf-core/seqtk/trim/main.nf` file and viewing the module metadata.
-
-```groovy title="modules/nf-core/seqtk/trim/main.nf" linenums="13"
-output:
-tuple val(meta), path("*.fastq.gz"), emit: reads
-path "versions.yml" , emit: versions
-```
-
-To help with organization and readability it is beneficial to create named output channels.
-
-For `SEQTK_TRIM`, the `reads` output could be put into a channel named `ch_trimmed`.
-
-```groovy title="workflows/myfirstpipeline.nf" linenums="35"
-ch_trimmed = SEQTK_TRIM.out.reads
-```
-
-Similarly, it is beneficial to immediately mix the tool versions into the `ch_versions` channel so they can be used as input for the `MULTIQC` process and passed to the final report.
-
-```groovy title="workflows/myfirstpipeline.nf" linenums="35"
-ch_versions = ch_versions.mix(SEQTK_TRIM.out.versions.first())
-```
-
-!!! note
-
- The `first` operator is used to emit the first item from `SEQTK_TRIM.out.versions` to avoid duplication.
-
-### Add a parameter to the `seqtk/trim` tool
-
-nf-core modules should be flexible and usable across many different pipelines. Therefore, tool parameters are typically not set in an nf-core/module. Instead, additional configuration options on how to run the tool, like its parameters or filename, can be applied to a module using the `conf/modules.config` file on the pipeline level. Process selectors (e.g., `withName`) are used to apply configuration options to modules selectively. Process selectors must be used within the `process` scope.
-
-The parameters or arguments of a tool can be changed using the directive `args`. You can find many examples of how arguments are added to modules in nf-core pipelines, for example, the nf-core/demo [modules.config](https://github.com/nf-core/demo/blob/master/conf/modules.config) file.
-
-Add this snippet to your `conf/modules.config` file (using the `params` scope) to call the `seqtk/trim` tool with the argument `-b 5` to trim 5 bp from the left end of each read:
-
-```console title="conf/modules.config" linenums="21"
-withName: 'SEQTK_TRIM' {
- ext.args = "-b 5"
-}
-```
-
-Run the pipeline again and check if the new parameter is applied:
-
-```bash
-nextflow run . -profile docker,test --outdir results
-
-[6c/34e549] process > MYORG_MYFIRSTPIPELINE:MYFIRSTPIPELINE:SEQTK_TRIM (SAMPLE1_PE) [100%] 3 of 3 ✔
-[27/397ccf] process > MYORG_MYFIRSTPIPELINE:MYFIRSTPIPELINE:MULTIQC [100%] 1 of 1 ✔
-```
-
-Copy the hash you see in your console output (here `6c/34e549`; it is different for _each_ run). You can `ls` using tab-completion in your `work` directory to expand the complete hash.
-In this folder you will find various log files. The `.command.sh` file contains the resolved command:
-
-```bash
-less work/6c/34e549912696b6757f551603d135bb/.command.sh
-```
-
-We can see, that the parameter `-b 5`, that we set in the `modules.config` is applied to the task:
-
-```console title="Output"
-#!/usr/bin/env bash
-
-set -e # Exit if a tool returns a non-zero status/exit code
-set -u # Treat unset variables and parameters as an error
-set -o pipefail # Returns the status of the last command to exit with a non-zero status or zero if all successfully execute
-set -C # No clobber - prevent output redirection from overwriting files.
-
-printf "%s\n" sample1_R1.fastq.gz sample1_R2.fastq.gz | while read f;
-do
- seqtk \
- trimfq \
- -b 5 \
- $f \
- | gzip --no-name > SAMPLE1_PE_$(basename $f)
-done
-
-cat <<-END_VERSIONS > versions.yml
-"MYORG_MYFIRSTPIPELINE:MYFIRSTPIPELINE:SEQTK_TRIM":
- seqtk: $(echo $(seqtk 2>&1) | sed 's/^.*Version: //; s/ .*$//')
-END_VERSIONS
-```
-
-### Takeaway
-
-You have now added a nf-core/module to your pipeline, configured it with a particular parameter, and made the output available in the workflow.
-
-### What's next?
-
-In the next step we will add a pipeline parameter to allow users to skip the trimming step.
-
----
-
-## Adding parameters to your pipeline
-
-Anything that a pipeline user may want to configure regularly should be made into a parameter so it can easily be overridden. nf-core defines some standards for providing parameters.
-
-Here, as a simple example, you will add a new parameter to your pipeline that will skip the `SEQTK_TRIM` process.
-
-Parameters are accessible in the pipeline script.
-
-### Default values
-
-In the nf-core template the default values for parameters are set in the `nextflow.config` in the base repository.
-
-Any new parameters should be added to the `nextflow.config` with a default value within the `params` scope.
-
-Parameter names should be unique and easily identifiable.
-
-We can a new parameter `skip_trim` to your `nextflow.config` file and set it to `false`.
-
-```groovy title="nextflow.config" linenums="16"
-// Trimming
-skip_trim = false
-```
-
-### Adding parameters to your pipeline
-
-Here, an `if` statement that is depended on the `skip_trim` parameter can be used to control the execution of the `SEQTK_TRIM` process. An `!` can be used to imply the logical "not".
-
-Thus, if the `skip_trim` parameter is **not** `true`, the `SEQTK_TRIM` will be be executed.
-
-```groovy title="workflows/myfirstpipeline.nf" linenums="29"
-//
-// MODULE: Run SEQTK_TRIM
-//
-if (!params.skip_trim) {
- SEQTK_TRIM (
- ch_samplesheet
- )
- ch_trimmed = SEQTK_TRIM.out.reads
- ch_versions = ch_versions.mix(SEQTK_TRIM.out.versions.first())
-}
-```
-
-Now your if statement has been added to your main workflow file and has a default setting in your `nextflow.config` file, you will be able to flexibly skip the new trimming step using the `skip_trim` parameter.
-
-We can now run the pipeline with the new `skip_trim` parameter to check it is working:
-
-```console
-nextflow run . -profile test,docker --outdir results --skip_trim
-```
-
-You should see that the `SEQTK_TRIM` process has been skipped in your execution:
-
-```console title="Output"
-!! Only displaying parameters that differ from the pipeline defaults !!
-------------------------------------------------------
-WARN: The following invalid input values have been detected:
-
-* --skip_trim: true
-
-
-executor > local (1)
-[7b/8b60a0] process > MYORG_MYFIRSTPIPELINE:MYFIRSTPIPELINE:MULTIQC [100%] 1 of 1 ✔
--[myorg/myfirstpipeline] Pipeline completed successfully-
-```
-
-### Validate input parameters
-
-When we ran the pipeline, we saw a warning message:
-
-```console
-WARN: The following invalid input values have been detected:
-
-* --skip_trim: true
-```
-
-Parameters are validated through the `nextflow_schema.json` file. This file is also used by the nf-core website (for example, in [nf-core/mag](https://nf-co.re/mag/3.2.1/parameters/)) to render the parameter documentation and print the pipeline help message (`nextflow run . --help`). If you have added parameters and they have not been documented in the `nextflow_schema.json` file, then the input validation does not recognize the parameter.
-
-The `nextflow_schema.json` file can get very big and very complicated very quickly.
-
-The `nf-core pipelines schema build` command is designed to support developers write, check, validate, and propose additions to your `nextflow_schema.json` file.
-
-```console
-nf-core pipelines schema build
-```
-
-It will enable you to launch a web builder to edit this file in your web browser rather than trying to edit this file manually.
-
-```console
-INFO [✓] Default parameters match schema validation
-INFO [✓] Pipeline schema looks valid (found 20 params)
-✨ Found 'params.skip_trim' in the pipeline config, but not in the schema. Add to pipeline schema? [y/n]: y
-INFO Writing schema with 21 params: 'nextflow_schema.json'
-🚀 Launch web builder for customization and editing? [y/n]: y
-```
-
-Using the web builder you can add add details about your new parameters.
-
-The parameters that you have added to your pipeline will be added to the bottom of the `nf-core pipelines schema build` file. Some information about these parameters will be automatically filled based on the default value from your `nextflow.config`. You will be able to categorize your new parameters into a group, add icons, and add descriptions for each.
-
-
-
-!!!note
-
- Ungrouped parameters in schema will cause a warning.
-
-Once you have made your edits you can click `Finished` and all changes will be automatically added to your `nextflow_schema.json` file.
-
-If you rerun the previous command, the warning should disappear:
-
-```console
-nextflow run . -profile test,docker --outdir results --skip_trim
-
-
-!! Only displaying parameters that differ from the pipeline defaults !!
-------------------------------------------------------
-executor > local (1)
-[6c/c78d0c] process > MYORG_MYFIRSTPIPELINE:MYFIRSTPIPELINE:MULTIQC [100%] 1 of 1 ✔
--[myorg/myfirstpipeline] Pipeline completed successfully-
-```
-
-### Takeaway
-
-You have added a new parameter to the pipeline, and learned how to use nf-core tools to describe it in the pipeline schema.
-
-### What's next?
-
-In the next step we will take a look at how we track metadata related to an input file.
-
----
-
-## Meta maps
-
-Datasets often contain additional information relevant to the analysis, such as a sample name, information about sequencing protocols, or other conditions needed in the pipeline to process certain samples together, determine their output name, or adjust parameters.
-
-By convention, nf-core tracks this information as `meta` maps. These are `key`-`value` pairs that are passed into modules together with the files. We already saw this briefly when inspecting the `input` for `seqtk`:
-
-```groovy title="modules/nf-core/seqtk/trim/main.nf" linenums="11"
-input:
-tuple val(meta), path(reads)
-```
-
-If we uncomment our earlier `view` statement:
-
-```groovy title="workflows/myfirstpipeline.nf" linenums="27"
-ch_samplesheet.view()
-```
-
-and run the pipeline again, we can see the current content of the `meta` maps:
-
-```console title="meta map"
-[[id:SAMPLE1_PE, single_end:false], ....]
-```
-
-You can add any field that you require to the `meta` map. By default, nf-core modules expect an `id` field.
-
-### Takeaway
-
-You know that a `meta` map is used to pass along additional information for a sample.
-
-### What's next?
-
-In the next step we will take a look how we can add a new key to the `meta` map using the samplesheet.
-
----
-
-## Simple Samplesheet adaptations
-
-nf-core pipelines typically use samplesheets as inputs to the pipelines. This allows us to:
-
-- validate each entry and print specific error messages.
-- attach information to each input file.
-- track which datasets are processed.
-
-Samplesheets are comma-separated text files with a header row specifying the column names, followed by one entry per row. For example, the samplesheet that we have been using during this teaching module looks like this:
-
-```csv title="samplesheet_test_illumina_amplicon.csv"
-sample,fastq_1,fastq_2
-SAMPLE1_PE,https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/illumina/amplicon/sample1_R1.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/illumina/amplicon/sample1_R2.fastq.gz
-SAMPLE2_PE,https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/illumina/amplicon/sample2_R1.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/illumina/amplicon/sample2_R2.fastq.gz
-SAMPLE3_SE,https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/illumina/amplicon/sample1_R1.fastq.gz,
-SAMPLE3_SE,https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/illumina/amplicon/sample2_R1.fastq.gz,
-```
-
-The structure of the samplesheet is specified in its own schema file in `assets/schema_input.json`. Each column has its own entry together with information about the column:
-
-```json title="schema_input.json"
-"properties": {
- "sample": {
- "type": "string",
- "pattern": "^\\S+$",
- "errorMessage": "Sample name must be provided and cannot contain spaces",
- "meta": ["id"]
- },
- "fastq_1": {
- "type": "string",
- "format": "file-path",
- "exists": true,
- "pattern": "^\\S+\\.f(ast)?q\\.gz$",
- "errorMessage": "FastQ file for reads 1 must be provided, cannot contain spaces and must have extension '.fq.gz' or '.fastq.gz'"
- },
- "fastq_2": {
- "type": "string",
- "format": "file-path",
- "exists": true,
- "pattern": "^\\S+\\.f(ast)?q\\.gz$",
- "errorMessage": "FastQ file for reads 2 cannot contain spaces and must have extension '.fq.gz' or '.fastq.gz'"
- }
-},
-"required": ["sample", "fastq_1"]
-```
-
-This validates that the samplesheet has at least two columns: `sample` and `fastq1` (`"required": ["sample", "fastq_1"]`). It also checks that `fastq1` and `fastq2` are files, and that the file endings match a particular pattern.
-Lastly, `sample` is information about the files that we want to attach and pass along the pipeline. nf-core uses `meta` maps for this: objects that have a key and a value. We can indicate this in the schema file directly by using the meta field:
-
-```json title="Sample column"
- "sample": {
- "type": "string",
- "pattern": "^\\S+$",
- "errorMessage": "Sample name must be provided and cannot contain spaces",
- "meta": ["id"]
- },
-```
-
-This sets the key name as `id` and the value that is in the `sample` column, for example `SAMPLE1_PE`:
-
-```console title="meta"
-[id: SAMPLE1_PE]
-```
-
-By adding a new entry into the JSON schema, we can attach additional meta information that we want to track. This will automatically validate it for us and add it to the meta map.
-
-Let's add some new meta information, like the `sequencer` as an optional column:
-
-```json title="assets/schema_input.json"
-"properties": {
- "sample": {
- "type": "string",
- "pattern": "^\\S+$",
- "errorMessage": "Sample name must be provided and cannot contain spaces",
- "meta": ["id"]
- },
- "sequencer": {
- "type": "string",
- "pattern": "^\\S+$",
- "meta": ["sequencer"]
- },
- "fastq_1": {
- "type": "string",
- "format": "file-path",
- "exists": true,
- "pattern": "^\\S+\\.f(ast)?q\\.gz$",
- "errorMessage": "FastQ file for reads 1 must be provided, cannot contain spaces and must have extension '.fq.gz' or '.fastq.gz'"
- },
- "fastq_2": {
- "type": "string",
- "format": "file-path",
- "exists": true,
- "pattern": "^\\S+\\.f(ast)?q\\.gz$",
- "errorMessage": "FastQ file for reads 2 cannot contain spaces and must have extension '.fq.gz' or '.fastq.gz'"
- }
-},
-"required": ["sample", "fastq_1"]
-```
-
-We can now run our normal tests with the old samplesheet:
-
-```console
-nextflow run . -profile docker,test --outdir results
-```
-
-The meta map now has a new key `sequencer`, that is empty because we did not specify a value yet:
-
-```console title="output"
-[['id':'SAMPLE1_PE', 'sequencer':[], 'single_end':false], ... ]
-[['id':'SAMPLE2_PE', 'sequencer':[], 'single_end':false], ... ]
-[['id':'SAMPLE3_SE', 'sequencer':[], 'single_end':true], ... ]
-```
-
-We have also prepared a new samplesheet, that has the `sequencer` column. You can overwrite the existing input with this command:
-
-```console
-nextflow run . -profile docker,test --outdir results --input ../data/sequencer_samplesheet.csv
-```
-
-This populates the `sequencer` and we can see it in the pipeline, when `view`ing the samplesheet channel:
-
-```console title="output"
-[['id':'SAMPLE1_PE', 'sequencer':'sequencer1', 'single_end':false], ... ]
-[['id':'SAMPLE2_PE', 'sequencer':'sequencer2', 'single_end':false], ... ]
-[['id':'SAMPLE3_SE', 'sequencer':'sequencer3', 'single_end':true], ... ]
-```
-
-We can comment the `ch_samplesheet.view()` line or remove it. We are not going to use it anymore in this training section.
-
-### Use the new meta key in the pipeline
-
-We can access this new meta value in the pipeline and use it to, for example, only enable trimming for samples from a particular sequencer. The [branch operator](https://www.nextflow.io/docs/stable/reference/operator.html#branch) let's us split
-an input channel into several new output channels based on a selection criteria:
-
-```groovy title="workflows/myfirstpipeline.nf" linenums="35"
-ch_seqtk_in = ch_samplesheet.branch { meta, reads ->
- to_trim: meta["sequencer"] == "sequencer2"
- other: true
-}
-
-SEQTK_TRIM (
- ch_seqtk_in.to_trim
-)
-```
-
-If we now rerun our default test, no reads are being trimmed (even though we did not specify `--skip_trim`):
-
-```console
-nextflow run . -profile docker,test --outdir results
-
-[- ] process > MYORG_MYFIRSTPIPELINE:MYFIRSTPIPELINE:SEQTK_TRIM -
-[5a/f580bc] process > MYORG_MYFIRSTPIPELINE:MYFIRSTPIPELINE:MULTIQC [100%] 1 of 1 ✔
-```
-
-If we use the samplesheet with the `sequencer` set, only one sample will be trimmed:
-
-```console
-nextflow run . -profile docker,test --outdir results --input ../data/sequencer_samplesheet.csv -resume
-
-[47/fdf9de] process > MYORG_MYFIRSTPIPELINE:MYFIRSTPIPELINE:SEQTK_TRIM (SAMPLE2_PE) [100%] 1 of 1 ✔
-[2a/a742ae] process > MYORG_MYFIRSTPIPELINE:MYFIRSTPIPELINE:MULTIQC [100%] 1 of 1 ✔
-```
-
-If you want to learn more about how to fine tune and develop the samplesheet schema further, visit [nf-schema](https://nextflow-io.github.io/nf-schema/2.2/nextflow_schema/sample_sheet_schema_specification/).
-
-### Takeaway
-
-You know how to adapt the samplesheet to add new meta information to your files.
-
-### What's next?
-
-In the next step we will add a module that is not yet in nf-core.
-
----
-
-## Create a custom module for your pipeline
-
-nf-core offers a comprehensive set of modules that have been created and curated by the community. However, as a developer, you may be interested in bespoke pieces of software that are not apart of the nf-core repository or customizing a module that already exists.
-
-In this instance, we will write a local module for the QC Tool [FastQE](https://fastqe.com/), which computes stats for FASTQ files and print those stats as emoji.
-
-This section should feel familiar to the `hello_modules` section.
-
-### Create the module
-
-!!! note "New module contributions are always welcome and encouraged!"
-
- If you have a module that you would like to contribute back to the community, reach out on the nf-core slack or open a pull request to the modules repository.
-
-Start by using the nf-core tooling to create a skeleton local module:
-
-```console
-nf-core modules create
-```
-
-It will ask you to enter the tool name and some configurations for the module. We will use the defaults here:
-
- - Specify the tool name: `Name of tool/subtool: fastqe`
- - Add the author name: `GitHub Username: (@):`
- - Accept the defaults for the remaining prompts by typing `enter`
-
-This will create a new file in `modules/local/fastqe.nf` that already contains the container and conda definitions, the general structure of the process, and a number of TODO statements to guide you through the adaptation.
-
-!!! warning
-
- If the module already exists locally, the command will fail to prevent you from accidentally overwriting existing work:
-
- ```console
- INFO Repository type: pipeline
- INFO Press enter to use default values (shown in brackets) or type your own responses. ctrl+click underlined text to open links.
- CRITICAL Module file exists already: 'modules/local/fastqe.nf'. Use '--force' to overwrite
- ```
-
-You will notice, that it still calls `samtools` and the input are `bam`.
-
-From our sample sheet, we know we have fastq files instead, so let's change the input definition accordingly:
-
-```groovy title="modules/local/fastqe.nf" linenums="38"
-tuple val(meta), path(reads)
-```
-
-The output of this tool is a tsv file with the emoji annotation, let's adapt the output as well:
-
-```groovy title="modules/local/fastqe.nf" linenums="42"
-tuple val(meta), path("*.tsv"), emit: tsv
-```
-
-The script section still calls `samtools`. Let's change this to the proper call of the tool:
-
-```groovy title="modules/local/fastqe.nf" linenums="62"
- fastqe \\
- $args \\
- $reads \\
- --output ${prefix}.tsv
-```
-
-And at last, we need to adapt the version retrieval. This tool does not have a version command, so we will add the release number manually:
-
-```groovy title="modules/local/fastqe.nf" linenums="52"
- def VERSION = '0.3.3'
-```
-
-and write it to a file in the script section:
-
-```groovy title="modules/local/fastqe.nf" linenums="70"
- fastqe: $VERSION
-```
-
-We will not cover [`stubs`](https://www.nextflow.io/docs/latest/process.html#stub) in this training. They are not necessary to run a module, so let's remove them for now:
-
-```groovy title="modules/local/fastqe.nf" linenums="74"
-stub:
- def args = task.ext.args ?: ''
- def prefix = task.ext.prefix ?: "${meta.id}"
- // TODO nf-core: A stub section should mimic the execution of the original module as best as possible
- // Have a look at the following examples:
- // Simple example: https://github.com/nf-core/modules/blob/818474a292b4860ae8ff88e149fbcda68814114d/modules/nf-core/bcftools/annotate/main.nf#L47-L63
- // Complex example: https://github.com/nf-core/modules/blob/818474a292b4860ae8ff88e149fbcda68814114d/modules/nf-core/bedtools/split/main.nf#L38-L54
- """
- touch ${prefix}.bam
-
- cat <<-END_VERSIONS > versions.yml
- "${task.process}":
- fastqe: \$(samtools --version |& sed '1!d ; s/samtools //')
- END_VERSIONS
- """
-```
-
-If you think this looks a bit messy and just want to add a complete final version, here's one we made earlier and we've removed all the commented out instructions:
-
-```groovy title="modules/local/fastqe.nf" linenums="1"
-process FASTQE {
- tag "$meta.id"
- label 'process_single'
-
- conda "${moduleDir}/environment.yml"
- container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
- 'https://depot.galaxyproject.org/singularity/fastqe:0.3.3--pyhdfd78af_0':
- 'biocontainers/fastqe:0.3.3--pyhdfd78af_0' }"
-
- input:
- tuple val(meta), path(reads)
-
- output:
- tuple val(meta), path("*.tsv"), emit: tsv
- path "versions.yml" , emit: versions
-
- when:
- task.ext.when == null || task.ext.when
-
- script:
- def args = task.ext.args ?: ''
- def prefix = task.ext.prefix ?: "${meta.id}"
- def VERSION = '0.3.3'
- """
- fastqe \\
- $args \\
- $reads \\
- --output ${prefix}.tsv
-
- cat <<-END_VERSIONS > versions.yml
- "${task.process}":
- fastqe: $VERSION
- END_VERSIONS
- """
-}
-```
-
-### Include the module into the pipeline
-
-The module is now ready in your `modules/local` folder, but not yet included in your pipeline. Similar to `seqtk/trim` we need to add it to `workflows/myfirstpipeline.nf`:
-
-_Before:_
-
-```groovy title="workflows/myfirstpipeline.nf" linenums="1"
-/*
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- IMPORT MODULES / SUBWORKFLOWS / FUNCTIONS
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-*/
-include { SEQTK_TRIM } from '../modules/nf-core/seqtk/trim/main'
-include { MULTIQC } from '../modules/nf-core/multiqc/main'
-```
-
-_After:_
-
-```groovy title="workflows/myfirstpipeline.nf" linenums="1"
-/*
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- IMPORT MODULES / SUBWORKFLOWS / FUNCTIONS
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-*/
-include { FASTQE } from '../modules/local/fastqe'
-include { SEQTK_TRIM } from '../modules/nf-core/seqtk/trim/main'
-include { MULTIQC } from '../modules/nf-core/multiqc/main'
-```
-
-and call it on our input data:
-
-```groovy title="workflows/myfirstpipeline.nf" linenums="47"
- FASTQE(ch_samplesheet)
- ch_versions = ch_versions.mix(FASTQE.out.versions.first())
-```
-
-Let's run the pipeline again:
-
-```console
-nextflow run . -profile docker,test --outdir results
-```
-
-In the results folder, you should now see a new subdirectory `fastqe/`, with the mean read qualities:
-
-```console title="SAMPLE1_PE.tsv"
-Filename Statistic Qualities
-sample1_R1.fastq.gz mean 😝 😝 😝 😝 😝 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😉 😉 😜 😜 😜 😉 😉 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😁 😉 😛 😜 😉 😉 😉 😉 😜 😜 😉 😉 😉 😉 😉 😁 😁 😁 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😜 😉 😉 😉 😉 😉 😜 😜 😜 😜 😜 😜 😜 😜 😜 😜 😜 😜 😜 😜 😛 😜 😜 😛 😛 😛 😚
-sample1_R2.fastq.gz mean 😌 😌 😌 😝 😝 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😜 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😉 😜 😉 😉 😜 😜 😉 😜 😜 😜 😜 😜 😜 😜 😜 😜 😜 😜 😜 😛 😜 😜 😜 😛 😜 😜 😜 😜 😛 😜 😛 😛 😛 😛 😛 😛 😛 😛 😛 😛 😛 😛 😝 😛 😝 😝 😝 😝 😝 😝 😝 😝 😝 😝 😝 😝 😝 😝 😌 😌 😌 😌 😌 😌 😌 😌 😌 😌 😌 😌 😌 😌 😌 😌 😌 😌 😌 😋 😋 😋 😋 😋 😋 😋 😋 😀
-```
-
-### Takeaway
-
-You know how to add a local module.
-
-And summarise your sequencing data as emojis.
-
----
-
-## Takeaway
-
-You know how to use the nf-core tooling to create a new pipeline, add modulea to it, apply tool and pipeline parameters, and adapt the samplesheet.
-
-## What's next?
-
-Celebrate and take another break! Next, we'll show you how to take advantage of Seqera Platform to launch and monitor your workflows more conveniently and efficiently on any compute infrastructure.
diff --git a/docs/nextflow_run/04_nf-core.md b/docs/nextflow_run/04_nf-core.md
new file mode 100644
index 000000000..6ae09f4a4
--- /dev/null
+++ b/docs/nextflow_run/04_nf-core.md
@@ -0,0 +1,7 @@
+# Part 3: Run nf-core
+
+1. Meet the nf-core style Hello World
+2. Run it (with test profile) and interpret the console output
+3. Locate the outputs (results)
+4. Find a 'real' nf-core pipeline
+5. Run nf-core/demo
diff --git a/docs/nextflow_run/04_run_seqera.md b/docs/nextflow_run/05_seqera.md
similarity index 87%
rename from docs/nextflow_run/04_run_seqera.md
rename to docs/nextflow_run/05_seqera.md
index 9da4c1d2b..0411ad37e 100644
--- a/docs/nextflow_run/04_run_seqera.md
+++ b/docs/nextflow_run/05_seqera.md
@@ -1,9 +1,4 @@
----
-title: "Part 9: Hello Seqera"
-description: Get started with Seqera Platform
----
-
-# Part 9: Hello Seqera
+# Part 4: Run on Seqera
So far we've been running Nextflow workflows on our local machine using the command line interface.
In this section, we'll introduce you to Seqera Platform, a powerful cloud-based platform for running, monitoring, and sharing Nextflow workflows.
diff --git a/docs/nextflow_run/img/DAG-preview.png b/docs/nextflow_run/img/DAG-preview.png
new file mode 100644
index 000000000..df76cc45e
Binary files /dev/null and b/docs/nextflow_run/img/DAG-preview.png differ
diff --git a/docs/nextflow_run/img/cpu-after.png b/docs/nextflow_run/img/cpu-after.png
deleted file mode 100644
index f173ef020..000000000
Binary files a/docs/nextflow_run/img/cpu-after.png and /dev/null differ
diff --git a/docs/nextflow_run/img/cpu-before.png b/docs/nextflow_run/img/cpu-before.png
deleted file mode 100644
index e0b8c76c8..000000000
Binary files a/docs/nextflow_run/img/cpu-before.png and /dev/null differ
diff --git a/docs/nextflow_run/img/dag-workflow.svg b/docs/nextflow_run/img/dag-workflow.svg
new file mode 100644
index 000000000..c5411bd34
--- /dev/null
+++ b/docs/nextflow_run/img/dag-workflow.svg
@@ -0,0 +1 @@
+
\ No newline at end of file
diff --git a/docs/nextflow_run/img/hello-pipeline-cowpy.svg b/docs/nextflow_run/img/hello-pipeline-cowpy.svg
new file mode 100644
index 000000000..09d97af72
--- /dev/null
+++ b/docs/nextflow_run/img/hello-pipeline-cowpy.svg
@@ -0,0 +1,5 @@
+
+
+
\ No newline at end of file
diff --git a/docs/nextflow_run/img/hello-pipeline-multi-inputs.svg b/docs/nextflow_run/img/hello-pipeline-multi-inputs.svg
new file mode 100644
index 000000000..9da92c273
--- /dev/null
+++ b/docs/nextflow_run/img/hello-pipeline-multi-inputs.svg
@@ -0,0 +1,4 @@
+
+
+
\ No newline at end of file
diff --git a/docs/nextflow_run/img/hello-pipeline-multi-steps.svg b/docs/nextflow_run/img/hello-pipeline-multi-steps.svg
new file mode 100644
index 000000000..5ff553a30
--- /dev/null
+++ b/docs/nextflow_run/img/hello-pipeline-multi-steps.svg
@@ -0,0 +1,4 @@
+
+
+
\ No newline at end of file
diff --git a/docs/nextflow_run/img/memory-after.png b/docs/nextflow_run/img/memory-after.png
deleted file mode 100644
index d61b4a7c5..000000000
Binary files a/docs/nextflow_run/img/memory-after.png and /dev/null differ
diff --git a/docs/nextflow_run/img/memory-before.png b/docs/nextflow_run/img/memory-before.png
deleted file mode 100644
index ce0f7ac27..000000000
Binary files a/docs/nextflow_run/img/memory-before.png and /dev/null differ
diff --git a/docs/nextflow_run/img/nested.excalidraw.svg b/docs/nextflow_run/img/nested.excalidraw.svg
deleted file mode 100644
index 277b24079..000000000
--- a/docs/nextflow_run/img/nested.excalidraw.svg
+++ /dev/null
@@ -1,21 +0,0 @@
-
\ No newline at end of file
diff --git a/docs/nextflow_run/img/nf-core-modules.png b/docs/nextflow_run/img/nf-core-modules.png
deleted file mode 100644
index 43c06de06..000000000
Binary files a/docs/nextflow_run/img/nf-core-modules.png and /dev/null differ
diff --git a/docs/nextflow_run/img/pipeline.excalidraw.svg b/docs/nextflow_run/img/pipeline.excalidraw.svg
deleted file mode 100644
index bcf2d6e54..000000000
--- a/docs/nextflow_run/img/pipeline.excalidraw.svg
+++ /dev/null
@@ -1,21 +0,0 @@
-
\ No newline at end of file
diff --git a/docs/nextflow_run/img/pipeline_schema.png b/docs/nextflow_run/img/pipeline_schema.png
deleted file mode 100644
index cae65fa48..000000000
Binary files a/docs/nextflow_run/img/pipeline_schema.png and /dev/null differ
diff --git a/docs/nextflow_run/img/report_cover.png b/docs/nextflow_run/img/report_cover.png
deleted file mode 100644
index 9feb1792c..000000000
Binary files a/docs/nextflow_run/img/report_cover.png and /dev/null differ
diff --git a/docs/nextflow_run/img/seqera-containers-1.png b/docs/nextflow_run/img/seqera-containers-1.png
deleted file mode 100644
index 37fe56c0c..000000000
Binary files a/docs/nextflow_run/img/seqera-containers-1.png and /dev/null differ
diff --git a/docs/nextflow_run/img/seqera-containers-2.png b/docs/nextflow_run/img/seqera-containers-2.png
deleted file mode 100644
index 2d2fb90b6..000000000
Binary files a/docs/nextflow_run/img/seqera-containers-2.png and /dev/null differ
diff --git a/docs/nextflow_run/img/with-collect-operator.svg b/docs/nextflow_run/img/with-collect-operator.svg
new file mode 100644
index 000000000..0b70c11d3
--- /dev/null
+++ b/docs/nextflow_run/img/with-collect-operator.svg
@@ -0,0 +1,4 @@
+
+
+
\ No newline at end of file
diff --git a/docs/nextflow_run/img/without-collect-operator.svg b/docs/nextflow_run/img/without-collect-operator.svg
new file mode 100644
index 000000000..fc2473ae0
--- /dev/null
+++ b/docs/nextflow_run/img/without-collect-operator.svg
@@ -0,0 +1,4 @@
+
+
+
\ No newline at end of file
diff --git a/docs/nextflow_run/index.md b/docs/nextflow_run/index.md
index 9b4c3837b..633639c3d 100644
--- a/docs/nextflow_run/index.md
+++ b/docs/nextflow_run/index.md
@@ -1,18 +1,16 @@
---
-title: Run Nextflow
+title: Nextflow Run
hide:
- toc
---
-# Run Nextflow
+# Nextflow Run
Hello! You are now on the path to running reproducible and scalable scientific workflows using Nextflow.
-[TODO] NEED TO DISTINGUISH CLEARLY FROM HELLO NEXTFLOW
-
The rise of big data has made it increasingly necessary to be able to analyze and perform experiments on large datasets in a portable and reproducible manner. Parallelization and distributed computing are the best ways to tackle this challenge, but the tools commonly available to computational scientists often lack good support for these techniques, or they provide a model that fits poorly with the needs of computational scientists. Nextflow was particularly created to address these challenges.
-During this training, you will be introduced to Nextflow in a series of complementary hands-on workshops.
+During this training, you will be introduced to Nextflow in a series of complementary hands-on tutorials.
Let's get started!
@@ -20,14 +18,15 @@ Let's get started!
## Learning objectives
-In this workshop, you will learn foundational concepts for building pipelines.
+In this workshop, you will learn foundational concepts and skills for configuring and running Nextflow pipelines without treating them like a black box.
By the end of this workshop you will be able to:
- Launch a Nextflow workflow locally
- Find and interpret outputs (results) and log files generated by Nextflow
- Troubleshoot basic issues
-- [TODO]
+- Identify the main components of a Nextflow workflow and explain at a general level how they relate to what the workflow does
+- Configure and manage the execution of Nextflow workflows
## Audience & prerequisites
@@ -36,4 +35,4 @@ This is a workshop for those who are completely new to Nextflow. Some basic fami
**Prerequisites**
- A GitHub account
-- Experience with command line
+- Basic familiarity with command line
diff --git a/mkdocs.yml b/mkdocs.yml
index acf974a30..7d59f8fd1 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -10,6 +10,12 @@ nav:
- envsetup/01_setup.md
- envsetup/02_local.md
- envsetup/03_devcontainer.md
+ - Nextflow Run:
+ - nextflow_run/index.md
+ - nextflow_run/00_orientation.md
+ - nextflow_run/01_basics.md
+ - nextflow_run/02_pipeline.md
+ - nextflow_run/03_config.md
- Hello Nextflow:
- hello_nextflow/index.md
- hello_nextflow/00_orientation.md
@@ -183,6 +189,7 @@ plugins:
- enumerate-headings:
restart_increment_after:
- envsetup/01_setup.md
+ - nextflow_run/00_orientation.md
- hello_nextflow/00_orientation.md
- hello_nf-core/00_orientation.md
- nf4_science/genomics/00_orientation.md
@@ -195,6 +202,7 @@ plugins:
- index*.md
- help*.md
- envsetup/*.md
+ - nextflow_run/*.md
- hello_nextflow/*.md
- hello_nf-core/*.md
- nf4_science/genomics/*.md
diff --git a/nextflow-run/1-hello.nf b/nextflow-run/1-hello.nf
new file mode 100644
index 000000000..74b4e75b3
--- /dev/null
+++ b/nextflow-run/1-hello.nf
@@ -0,0 +1,26 @@
+#!/usr/bin/env nextflow
+
+/*
+ * Use echo to print 'Hello World!' to a file
+ */
+process sayHello {
+
+ publishDir 'results', mode: 'copy'
+
+ input:
+ val greeting
+
+ output:
+ path 'output.txt'
+
+ script:
+ """
+ echo '$greeting' > output.txt
+ """
+}
+
+workflow {
+
+ // emit a greeting
+ sayHello(params.greeting)
+}
diff --git a/nextflow-run/2a-inputs.nf b/nextflow-run/2a-inputs.nf
new file mode 100644
index 000000000..03739b775
--- /dev/null
+++ b/nextflow-run/2a-inputs.nf
@@ -0,0 +1,31 @@
+#!/usr/bin/env nextflow
+
+/*
+ * Use echo to print 'Hello World!' to a file
+ */
+process sayHello {
+
+ publishDir 'results', mode: 'copy'
+
+ input:
+ val greeting
+
+ output:
+ path "${greeting}-output.txt"
+
+ script:
+ """
+ echo '$greeting' > '$greeting-output.txt'
+ """
+}
+
+workflow {
+
+ // create a channel for inputs from a CSV file
+ greeting_ch = Channel.fromPath(params.input)
+ .splitCsv()
+ .map { line -> line[0] }
+
+ // emit a greeting
+ sayHello(greeting_ch)
+}
diff --git a/nextflow-run/2b-multistep.nf b/nextflow-run/2b-multistep.nf
new file mode 100644
index 000000000..386bafad0
--- /dev/null
+++ b/nextflow-run/2b-multistep.nf
@@ -0,0 +1,75 @@
+#!/usr/bin/env nextflow
+
+/*
+ * Use echo to print 'Hello World!' to a file
+ */
+process sayHello {
+
+ publishDir 'results', mode: 'copy'
+
+ input:
+ val greeting
+
+ output:
+ path "${greeting}-output.txt"
+
+ script:
+ """
+ echo '$greeting' > '$greeting-output.txt'
+ """
+}
+
+/*
+ * Use a text replacement tool to convert the greeting to uppercase
+ */
+process convertToUpper {
+
+ publishDir 'results', mode: 'copy'
+
+ input:
+ path input_file
+
+ output:
+ path "UPPER-${input_file}"
+
+ script:
+ """
+ cat '$input_file' | tr '[a-z]' '[A-Z]' > 'UPPER-${input_file}'
+ """
+}
+
+/*
+ * Collect uppercase greetings into a single output file
+ */
+process collectGreetings {
+
+ publishDir 'results', mode: 'copy'
+
+ input:
+ path input_files
+
+ output:
+ path "COLLECTED-output.txt"
+
+ script:
+ """
+ cat ${input_files} > 'COLLECTED-output.txt'
+ """
+}
+
+workflow {
+
+ // create a channel for inputs from a CSV file
+ greeting_ch = Channel.fromPath(params.input)
+ .splitCsv()
+ .map { line -> line[0] }
+
+ // emit a greeting
+ sayHello(greeting_ch)
+
+ // convert the greeting to uppercase
+ convertToUpper(sayHello.out)
+
+ // collect all the greetings into one file
+ collectGreetings(convertToUpper.out.collect())
+}
diff --git a/nextflow-run/2c-modules.nf b/nextflow-run/2c-modules.nf
new file mode 100644
index 000000000..f44f08738
--- /dev/null
+++ b/nextflow-run/2c-modules.nf
@@ -0,0 +1,31 @@
+#!/usr/bin/env nextflow
+
+/*
+ * Pipeline parameters
+ */
+params.input = 'greetings.csv'
+
+// Include modules
+include { sayHello } from './modules/sayHello.nf'
+include { convertToUpper } from './modules/convertToUpper.nf'
+include { collectGreetings } from './modules/collectGreetings.nf'
+
+workflow {
+
+ // create a channel for inputs from a CSV file
+ greeting_ch = Channel.fromPath(params.input)
+ .splitCsv()
+ .map { line -> line[0] }
+
+ // emit a greeting
+ sayHello(greeting_ch)
+
+ // convert the greeting to uppercase
+ convertToUpper(sayHello.out)
+
+ // collect all the greetings into one file
+ collectGreetings(convertToUpper.out.collect())
+
+ // emit a message about the size of the batch
+ sayHello.out.count().view { "There were $it greetings in this batch" }
+}
diff --git a/nextflow-run/2d-container.nf b/nextflow-run/2d-container.nf
new file mode 100644
index 000000000..e8c6b1caa
--- /dev/null
+++ b/nextflow-run/2d-container.nf
@@ -0,0 +1,27 @@
+#!/usr/bin/env nextflow
+
+// Include modules
+include { sayHello } from './modules/sayHello.nf'
+include { convertToUpper } from './modules/convertToUpper.nf'
+include { collectGreetings } from './modules/collectGreetings.nf'
+include { cowpy } from './modules/cowpy.nf'
+
+workflow {
+
+ // create a channel for inputs from a CSV file
+ greeting_ch = Channel.fromPath(params.input)
+ .splitCsv()
+ .map { line -> line[0] }
+
+ // emit a greeting
+ sayHello(greeting_ch)
+
+ // convert the greeting to uppercase
+ convertToUpper(sayHello.out)
+
+ // collect all the greetings into one file
+ collectGreetings(convertToUpper.out.collect())
+
+ // generate ASCII art with cowpy
+ cowpy(collectGreetings.out, params.character)
+}
diff --git a/nextflow-run/3-main.nf b/nextflow-run/3-main.nf
new file mode 100644
index 000000000..e4d4acc06
--- /dev/null
+++ b/nextflow-run/3-main.nf
@@ -0,0 +1,30 @@
+#!/usr/bin/env nextflow
+
+// Default parameters
+params.input = 'greetings.csv'
+params.character = 'turkey'
+// Include modules
+include { sayHello } from './modules/sayHello.nf'
+include { convertToUpper } from './modules/convertToUpper.nf'
+include { collectGreetings } from './modules/collectGreetings.nf'
+include { cowpy } from './modules/cowpy.nf'
+
+workflow {
+
+ // create a channel for inputs from a CSV file
+ greeting_ch = Channel.fromPath(params.input)
+ .splitCsv()
+ .map { line -> line[0] }
+
+ // emit a greeting
+ sayHello(greeting_ch)
+
+ // convert the greeting to uppercase
+ convertToUpper(sayHello.out)
+
+ // collect all the greetings into one file
+ collectGreetings(convertToUpper.out.collect())
+
+ // generate ASCII art with cowpy
+ cowpy(collectGreetings.out, params.character)
+}
diff --git a/nextflow-run/greetings.csv b/nextflow-run/greetings.csv
new file mode 100644
index 000000000..8f156a346
--- /dev/null
+++ b/nextflow-run/greetings.csv
@@ -0,0 +1,3 @@
+Hello,English,123
+Bonjour,French,456
+Holà,Spanish,789
diff --git a/nextflow-run/modules/collectGreetings.nf b/nextflow-run/modules/collectGreetings.nf
new file mode 100644
index 000000000..01ffb28a8
--- /dev/null
+++ b/nextflow-run/modules/collectGreetings.nf
@@ -0,0 +1,18 @@
+/*
+ * Collect uppercase greetings into a single output file
+ */
+process collectGreetings {
+
+ publishDir 'results', mode: 'copy'
+
+ input:
+ path input_files
+
+ output:
+ path "COLLECTED-output.txt"
+
+ script:
+ """
+ cat ${input_files} > 'COLLECTED-output.txt'
+ """
+}
diff --git a/nextflow-run/modules/convertToUpper.nf b/nextflow-run/modules/convertToUpper.nf
new file mode 100644
index 000000000..b2689e8e9
--- /dev/null
+++ b/nextflow-run/modules/convertToUpper.nf
@@ -0,0 +1,20 @@
+#!/usr/bin/env nextflow
+
+/*
+ * Use a text replacement tool to convert the greeting to uppercase
+ */
+process convertToUpper {
+
+ publishDir 'results', mode: 'copy'
+
+ input:
+ path input_file
+
+ output:
+ path "UPPER-${input_file}"
+
+ script:
+ """
+ cat '$input_file' | tr '[a-z]' '[A-Z]' > 'UPPER-${input_file}'
+ """
+}
diff --git a/nextflow-run/modules/cowpy.nf b/nextflow-run/modules/cowpy.nf
new file mode 100644
index 000000000..05ff5ec57
--- /dev/null
+++ b/nextflow-run/modules/cowpy.nf
@@ -0,0 +1,22 @@
+#!/usr/bin/env nextflow
+
+// Generate ASCII art with cowpy
+process cowpy {
+
+ container 'community.wave.seqera.io/library/cowpy:1.1.5--3db457ae1977a273'
+
+ publishDir 'results', mode: 'copy'
+
+ input:
+ path input_file
+ val character
+
+ output:
+ path "cowpy-${input_file}"
+
+ script:
+ """
+ cat $input_file | cowpy -c "$character" > cowpy-${input_file}
+ """
+
+}
diff --git a/nextflow-run/modules/sayHello.nf b/nextflow-run/modules/sayHello.nf
new file mode 100644
index 000000000..6005ad54c
--- /dev/null
+++ b/nextflow-run/modules/sayHello.nf
@@ -0,0 +1,20 @@
+#!/usr/bin/env nextflow
+
+/*
+ * Use echo to print 'Hello World!' to a file
+ */
+process sayHello {
+
+ publishDir 'results', mode: 'copy'
+
+ input:
+ val greeting
+
+ output:
+ path "${greeting}-output.txt"
+
+ script:
+ """
+ echo '$greeting' > '$greeting-output.txt'
+ """
+}
diff --git a/nextflow-run/nextflow.config b/nextflow-run/nextflow.config
new file mode 100644
index 000000000..d3af3eaae
--- /dev/null
+++ b/nextflow-run/nextflow.config
@@ -0,0 +1 @@
+docker.enabled = true
diff --git a/nextflow-run/test-params.yaml b/nextflow-run/test-params.yaml
new file mode 100644
index 000000000..c142eaf19
--- /dev/null
+++ b/nextflow-run/test-params.yaml
@@ -0,0 +1,2 @@
+input: "greetings.csv"
+character: "stegosaurus"