Update docs

Signed-off-by: Ben Sherman <[email protected]>

Author: bentsherman

docs/aws.md

@@ -550,31 +550,3 @@ To enable this capability, you need to enable the Wave service in the Nextflow c
 ## Advanced configuration
 
 Read the {ref}`AWS configuration<config-aws>` section to learn more about advanced configuration options.
-
-## AWS SDK v1 to v2 migration
-
-AWS Java SDK v1 is reaching end of life in December 2025. Since version `25.06.0-edge`, Nextflow uses AWS Java SDK v2 in the `nf-amazon` core plugin.
-The migration to the new version has introduced significant changes in the plugin. This section provides an overview of how these changes can affect the Nextflow configuration.
-
-### HTTP Client
-
-The most important change is in the HTTP clients. In v1, there was a single synchronous HTTP client to interact with AWS services.
-In v2, AWS Java SDK users can choose between different sync and async clients.
-To reduce the number of changes, the `nf-amazon` core plugin uses the HTTP sync client as in v1 in most cases, except for S3 transfers managed by the S3 Transfer Manager, where an async client is required.
-For these transfers, the plugin uses the CRT-based S3 async client, which is recommended for performance in large transfers.
-
-### Signer and User Agent
-
-Another important change in SDK v2 is the supported request signer. The new SDK only implements the AWS Signer V4, and the CRT client does not support overriding advanced HTTP options such as the signer override or user agent prefix.
-For this reason, overriding the signer or defining a user agent is no longer supported.
-
-### Nextflow configuration
-
-As a consequence of the changes mentioned above, the supported `aws.client` configuration options have been modified as described below.
-
-* New options have been added to tune the new S3 Transfer Manager and CRT-based client. The main CRT-based client configuration is `targetThroughputInGbps`. This is used to automatically set low-level parameters like `maxConcurrency`, `maxNativeMemory`, `minimumPartSize`, and `multipartThreshold`.
-  You can also define `transferManagerThreads`, but note this does not limit transfer concurrency — it is used for auxiliary tasks such as traversing directories, etc.
-
-* Options `signerOverride`, `userAgent`, `protocol`, `socketSendBufferSizeHint`, and `socketRecvBufferSizeHint` are no longer supported by AWS SDK v2.
-
-* Multi-part uploads are automatically managed by the S3 Transfer Manager in most cases. Therefore, the options `uploadChunkSize`, `uploadMaxAttempts`, `uploadMaxThreads`, and `uploadRetrySleep` are only used in the `Files.newOutputStream()` case, which cannot be handled by the S3 Transfer Manager.

docs/conf.py

@@ -50,7 +50,8 @@
     'mail.md': 'notifications.md',
     'operator.md': 'reference/operator.md',
     'dsl1.md': 'migrations/dsl1.md',
-    'updating-syntax.md': 'strict-syntax.md'
+    'updating-syntax.md': 'strict-syntax.md',
+    'updating-spot-retries.md': 'guides/updating-spot-retries.md'
 }
 
 # Add any paths that contain templates here, relative to this directory.

docs/guides/aws-java-sdk-v2.md

@@ -0,0 +1,34 @@
+(aws-java-sdk-v2-page)=
+
+# AWS Java SDK v2
+
+AWS Java SDK v1 is reaching end of life at the end of 2025. Starting in version `25.06.0-edge`, Nextflow uses AWS Java SDK v2 in the `nf-amazon` plugin.
+
+This migration introduced several breaking changes to the `aws.client` config scope, including new options and removed options. This page describes these changes and how they affect your Nextflow configuraiton.
+
+## New HTTP client
+
+The HTTP client used by SDK v2 does not support overriding certain advanced HTTP options. As a result, the following config options are no longer supported:
+
+- `aws.client.protocol`
+- `aws.client.signerOverride`
+- `aws.client.socketRecvBufferSizeHint`
+- `aws.client.socketSendBufferSizeHint`
+- `aws.client.userAgent`
+
+## S3 transfer manager
+
+The *S3 transfer manager* is a subsystem of SDK v2 which handles S3 transfers, including S3 uploads and downloads.
+
+The concurrency and throughput of the S3 transfer manager can be configured manually using the `aws.client.maxConcurrency` and `aws.client.maxNativeMemory` config options. Alternatively, the `aws.client.targetThroughputInGbps` config option can be used to set the previous two options automatically based on a target throughput.
+
+## Multi-part uplaods
+
+Multi-part uploads are handled by the S3 transfer manager. The `aws.client.minimumPartSize` and `aws.client.multipartThreshold` config options can be used to control when and how multi-part uploads are performed.
+
+The following multi-part upload options are no longer supported:
+
+- `aws.client.uploadChunkSize`
+- `aws.client.uploadMaxAttempts`
+- `aws.client.uploadMaxThreads`
+- `aws.client.uploadRetrySleep`

docs/updating-spot-retries.md renamed to docs/guides/updating-spot-retries.md

@@ -158,11 +158,19 @@ developer/plugins
 
 ```{toctree}
 :hidden:
-:caption: Guides
+:caption: Tutorials
 :maxdepth: 1
 
 data-lineage
-updating-spot-retries
 metrics
 flux
 ```
+
+```{toctree}
+:hidden:
+:caption: Guides
+:maxdepth: 1
+
+guides/aws-java-sdk-v2
+guides/updating-spot-retries
+```

docs/index.md

@@ -0,0 +1,13 @@
+(migrating-25-10-page)=
+
+# Migrating to 25.10 (preview)
+
+This page summarizes the upcoming changes in Nextflow 25.10, which will be released in October 2025.
+
+:::{note}
+This page is a work in progress and will be updated as features are finalized. It should not be considered complete until the 25.10 release.
+:::
+
+## Breaking changes
+
+- The AWS Java SDK used by Nextflow was upgraded from v1 to v2, which introduced some breaking changes to the `aws.client` config options. See {ref}`the guide <aws-java-sdk-v2-page>` for details.

docs/migrations/25-10.md

@@ -222,28 +222,28 @@ The following settings are available:
 `aws.client.maxConcurrency`
 : :::{versionadded} 25.06.0-edge
   :::
-: The maximum number of concurrent S3 transfers in the S3 Transfer Manager's client. By default, this setting is determined by `aws.client.targetThroughputInGbps`. Modifying this value can affect the amount of memory used for S3 transfers.
+: The maximum number of concurrent S3 transfers used by the S3 transfer manager. By default, this setting is determined by `aws.client.targetThroughputInGbps`. Modifying this value can affect the amount of memory used for S3 transfers.
 
 `aws.client.maxConnections`
-: The maximum number of allowed open HTTP connections when using the synchronous S3 client (default: `50`).
+: The maximum number of open HTTP connections used by the S3 transfer manager (default: `50`).
 
 `aws.client.maxErrorRetry`
 : The maximum number of retry attempts for failed retryable requests (default: `-1`).
 
 `aws.client.maxNativeMemory`
 : :::{versionadded} 25.06.0-edge
   :::
-: The maximum native memory used by the S3 Transfers Manager's client. By default, this setting is determined by `aws.client.targetThroughputInGbps`. Modifying this value can affect the memory used by the S3 transfers.
+: The maximum native memory used by the S3 transfer manager. By default, this setting is determined by `aws.client.targetThroughputInGbps`.
 
 `aws.client.minimumPartSize`
 : :::{versionadded} 25.06.0-edge
   :::
-: The minimum part size used by the S3 Transfer Manager's client multi-part uploads (default: `8 MB`).
+: The minimum part size used by the S3 transfer manager for multi-part uploads (default: `8 MB`).
 
 `aws.client.multipartThreshold`
 : :::{versionadded} 25.06.0-edge
   :::
-: The object size threshold for performing multi-part uploads in the S3 Transfer Manager's client (default: same as `aws.cllient.minimumPartSize`).
+: The object size threshold used by the S3 transfer manager for performing multi-part uploads (default: same as `aws.cllient.minimumPartSize`).
 
 `aws.client.protocol`
 : :::{deprecated} 25.06.0-edge
@@ -308,7 +308,7 @@ The following settings are available:
 `aws.client.targetThroughputInGbps`
 : :::{versionadded} 25.06.0-edge
   :::
-: The target network throughput (in Gbps) when using the asynchronous S3 client (default: `10`). This setting is not used when `aws.client.maxConcurrency` and `aws.client.maxNativeMemory` are specified.
+: The target network throughput (in Gbps) used by the S3 transfer manager (default: `10`). This setting is not used when `aws.client.maxConcurrency` and `aws.client.maxNativeMemory` are specified.
 
 `aws.client.transferManagerThreads`
 : :::{versionadded} 25.06.0-edge
@@ -339,14 +339,12 @@ The following settings are available:
   :::
 : The maximum number of threads used for multipart upload (default: `10`).
 
-
 `aws.client.uploadRetrySleep`
 : :::{deprecated} 25.06.0-edge
   This option is no longer supported.
   :::
 : The time to wait after a failed upload attempt to retry the part upload (default: `500ms`).
 
-
 `aws.client.uploadStorageClass`
 : The S3 storage class applied to stored objects. Can be `STANDARD`, `STANDARD_IA`, `ONEZONE_IA`, or `INTELLIGENT_TIERING` (default: `STANDARD`).
 

docs/reference/config.md

@@ -85,12 +85,6 @@ The minimum size of a single part in a multipart upload (default: `8 MB`).
     """)
     public MemoryUnit multipartThreshold;
 
-    @ConfigOption
-    @Description("""
-        The protocol (i.e. HTTP or HTTPS) to use when connecting to AWS.
-    """)
-    public String protocol;
-
     @ConfigOption
     @Description("""
         The proxy host to connect through.
@@ -133,24 +127,6 @@ The protocol scheme to use when connecting through a proxy (http/https).
     """)
     public boolean s3PathStyleAccess;
 
-    @ConfigOption
-    @Description("""
-        The name of the signature algorithm to use for signing requests made by the client.
-    """)
-    public String signerOverride;
-
-    @ConfigOption
-    @Description("""
-        The size hint (in bytes) for the low level TCP send buffer.
-    """)
-    public int socketSendBufferSizeHint;
-
-    @ConfigOption
-    @Description("""
-        The size hint (in bytes) for the low level TCP receive buffer.
-    """)
-    public int socketRecvBufferSizeHint;
-
     @ConfigOption
     @Description("""
         The amount of time to wait (in milliseconds) for data to be transferred over an established, open connection before the connection is timed out.
@@ -181,36 +157,6 @@ The number of threads used by the S3 transfer manager (default: `10`).
     """)
     public int transferManagerThreads;
 
-    @ConfigOption
-    @Description("""
-        The HTTP user agent header passed with all HTTP requests.
-    """)
-    public String userAgent;
-
-    @ConfigOption
-    @Description("""
-        The size of a single part in a multipart upload (default: `100 MB`).
-    """)
-    public MemoryUnit uploadChunkSize;
-
-    @ConfigOption
-    @Description("""
-        The maximum number of upload attempts after which a multipart upload returns an error (default: `5`).
-    """)
-    public int uploadMaxAttempts;
-
-    @ConfigOption
-    @Description("""
-        The maximum number of threads used for multipart upload.
-    """)
-    public int uploadMaxThreads;
-
-    @ConfigOption
-    @Description("""
-        The time to wait after a failed upload attempt to retry the part upload (default: `500ms`).
-    """)
-    public Duration uploadRetrySleep;
-
     @ConfigOption
     @Description("""
         The S3 storage class applied to stored objects, one of \\[`STANDARD`, `STANDARD_IA`, `ONEZONE_IA`, `INTELLIGENT_TIERING`\\] (default: `STANDARD`).