Skip to content

Allow docker registry to override fully-qualified container images #6178

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 24 additions & 1 deletion docs/reference/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -589,7 +589,30 @@ The following settings are available:
: Add the specified flags to the volume mounts e.g. `mountFlags = 'ro,Z'`.

`docker.registry`
: The registry from where Docker images are pulled. It should be only used to specify a private registry server. It should NOT include the protocol prefix i.e. `http://`.
: Specify a custom registry from which to pull Docker images. It should be only used to specify a private registry server. It should NOT include the protocol prefix i.e. `http://`.
: The custom registry is only applied to container images that don't specify a registry. Use `docker.registryOverrides` to also override fully-qualified container images.

`docker.registryOverrides`
: :::{versionadded} 25.06.0-edge
:::
: A list of registries to override when specifying a custom registry with `docker.registry`. For example:

```groovy
docker.registryOverrides = [
'community.wave.seqera.io',
'quay.io'
]
```

: Nextflow will use this list to modify fully-qualified container images to use the registry specified by `docker.registry`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
: Nextflow will use this list to modify fully-qualified container images to use the registry specified by `docker.registry`.
: Nextflow will use this list to modify fully-qualified container images and use the registry specified by `docker.registry`.

Please check meaning is retained


: :::{tip}
Use `nextflow inspect` to obtain the list of all containers used by a pipeline, from which you can determine the set of registries that need to be overridden.
Copy link
Collaborator

@christopher-hakkaart christopher-hakkaart Jun 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Use `nextflow inspect` to obtain the list of all containers used by a pipeline, from which you can determine the set of registries that need to be overridden.
To determine the set of registries that need to be overridden, use `nextflow inspect` to obtain the list of all containers used by a pipeline.

:::

: :::{warning}
The pipeline will fail if a container image cannot be overridden when using this setting, in order to prevent Nextflow from pulling containers from outside the custom registry.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The pipeline will fail if a container image cannot be overridden when using this setting, in order to prevent Nextflow from pulling containers from outside the custom registry.
To prevent pulling containers from outside the custom registry, Nextflow will fail the pipeline if it cannot override a container image when using this setting.

Please check the meaning is retained.

:::

`docker.remove`
: Clean-up the container after the execution (default: `true`). See the [Docker documentation](https://docs.docker.com/engine/reference/run/#clean-up---rm) for details.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -157,17 +157,36 @@ class ContainerHandler {
if( !imageName )
return null

String reg = this.config?.registry
if( !reg )
final registry = normalizeRegistry(this.config?.registry as String)
final registryOverrides = (this.config?.registryOverrides as List<String>)?.collect((r) -> normalizeRegistry(r))
if( !registry )
return imageName

if( isAbsoluteDockerName(imageName) )
// apply the registry only to relative container images
// when registry overrides are not specified
if( !isAbsoluteDockerName(imageName) )
return registry + imageName

if( !registryOverrides )
return imageName

if( !reg.endsWith('/') )
reg += '/'
// when registry overrides are specified, apply the custom registry also
// to fully-qualified container images
final newImageName = overrideRegistry(imageName, registry, registryOverrides)
if( newImageName )
return newImageName

// otherwise, raise an error to prevent Nextflow from pulling container images
// from outside of the custom registry
throw new IllegalArgumentException("Registry for container image '$imageName' not found in overrides")
}

return reg + imageName
static String normalizeRegistry(String registry) {
if( !registry )
return null
return registry.endsWith('/')
? registry
: registry + '/'
}

static boolean isAbsoluteDockerName(String image) {
Expand All @@ -179,6 +198,14 @@ class ContainerHandler {
image.contains('.') || image.contains(':')
}

static String overrideRegistry(String imageName, String toRegistry, List<String> registryOverrides) {
for( final fromRegistry : registryOverrides ) {
if( imageName.startsWith(fromRegistry) )
return toRegistry + imageName.substring(fromRegistry.length())
}
return null
}

/**
* Normalize Udocker image name adding `:latest`
* when required
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,26 @@ class ContainerHandlerTest extends Specification {

}

def 'should override absolute docker image name when enabled' () {

given:
def registry = 'd.reg'
def registryOverrides = [ 'registry:5000' ]
def handler = new ContainerHandler([registry: registry, registryOverrides: registryOverrides])

when:
def result = handler.normalizeDockerImageName('registry:5000/cbcrg/hello')
then:
result == 'd.reg/cbcrg/hello'

when:
handler.normalizeDockerImageName('registry:8000/cbcrg/hello')
then:
def e = thrown(IllegalArgumentException)
e.message == "Registry for container image 'registry:8000/cbcrg/hello' not found in overrides"

}

def 'test normalize shifter image name' () {

given:
Expand Down
Loading