Add Split Segments Toolkit and Draw Mode for Skeleton Tool

CHANGELOG.unreleased.md

@@ -11,6 +11,9 @@ For upgrade instructions, please check the [migration guide](MIGRATIONS.released
 [Commits](https://github.com/scalableminds/webknossos/compare/25.04.0...HEAD)
 
 ### Added
+- Added a new "draw" mode for the skeleton tool. When enabled, one can rapidly create notes by keeping the left mouse button pressed and moving the cursor. [#8434](https://github.com/scalableminds/webknossos/pull/8434)
+- Added the concept of "toolkits". By default, all tools are available to the user (as before), but one can select a specific toolkit to only see certain tools. Some toolkits also change the behavior of the tools. For example, the "Split Segments" toolkit (see below). [#8434](https://github.com/scalableminds/webknossos/pull/8434)
+- Added a workflow for splitting segments. Select the "Split Segments" toolkit and create a bounding box in which you want to execute the split. Then, use the skeleton tool to place nodes on the boundary between two (merged) segments. The nodes will be used to construct a 3D surface. Finally, use the floodfill tool (enable 3D and bounding-box restriction) to relabel a part of the segment. the floodfill won't cross the 3D surface. [#8434](https://github.com/scalableminds/webknossos/pull/8434)
 - Added more layer specific settings to the configurations included in sharing links. [#8539](https://github.com/scalableminds/webknossos/pull/8539)
 - When uploading multiple NMLs at once, the description is now kept, if all NMLs with non-empty descriptions have the same description. [#8533](https://github.com/scalableminds/webknossos/pull/8533)
 

README.md

@@ -2,7 +2,7 @@
 <img align="right" src="https://raw.githubusercontent.com/scalableminds/webknossos/master/public/images/logo-icon-only.svg" alt="WEBKNOSSOS Logo" width="150" />
 WEBKNOSSOS is an open-source tool for annotating and exploring large 3D image datasets.
 
-* Fly through your data for fast skeletonization and proof-reading
+* Fly through your data for fast skeletonization and proofreading
 * Create 3D training data for automated segmentations efficiently
 * Scale data reconstruction projects with crowdsourcing workflows
 * Share datasets and annotations with collaborating scientists
@@ -25,7 +25,7 @@ WEBKNOSSOS is an open-source tool for annotating and exploring large 3D image da
 * Optimized performance for large annotations
 * User and task management for high-throughput crowdsourcing
 * Sharing and collaboration features
-* Proof-Reading tools for working with large (over)-segmentations
+* Proofreading tools for working with large (over)-segmentations
 * [Standalone datastore component](https://github.com/scalableminds/webknossos/tree/master/webknossos-datastore) for flexible deployments
 * Supported dataset formats: [WKW](https://github.com/scalableminds/webknossos-wrap), [Neuroglancer Precomputed](https://github.com/google/neuroglancer/tree/master/src/datasource/precomputed), [Zarr](https://zarr.dev), [N5](https://github.com/saalfeldlab/n5)
 * Supported image formats: Grayscale, Segmentation Maps, RGB, Multi-Channel

docs/getting_started.md

@@ -82,7 +82,7 @@ Now you know the basics of WEBKNOSSOS.
 Feel free to explore more features of WEBKNOSSOS in this documentation.
 
 - [Dashboard](./dashboard/index.md)
-- [Volume Annotations & Proof-Reading](./volume_annotation/index.md)
+- [Volume Annotations & Proofreading](./volume_annotation/index.md)
 - [Skeleton Annotations](./skeleton_annotation/index.md)
 - [Understanding the User Interface](./ui/index.md)
 - [Keyboard Shortcuts](./ui/keyboard_shortcuts.md)

docs/proofreading/index.md

@@ -4,6 +4,6 @@ WEBKNOSSOS offers several ways for proofreading large-scale, volumetric segmenta
 
 There are three proofreading workflows supported by WEBKNOSSOS:
 
-1. The [Proofreading tool](tools.md) to correct large segmentations based on an underlying super-voxel graph. To use the proofreading tool, you need a [segmentation mapping](segmentation_mappings.md).
-2. The [Merger-Mode tool](merger_mode.md)
-
+1. The [Proofreading tool](proofreading_tool.md) to correct split and merge errors in large segmentations based on an underlying super-voxel graph. To use the proofreading tool, you need a [segmentation mapping](segmentation_mappings.md).
+2. The [Merger-Mode tool](merger_mode.md) which allows merging of segments by constructing a lightweight skeleton-based mapping.
+3. Voxel-based relabeling of segments. Segments can be merged using the fill tool. To split segments there is a dedicated [Split Segments toolkit](split_segments_toolkit.md).

docs/proofreading/split_segments_toolkit.md

@@ -0,0 +1,31 @@
+# Split Segments Toolkit
+
+As explained in the [Toolkits](../ui/toolbar.md#Toolkits) section, there is a dedicated toolkit for splitting segments in your volume annotation.
+With this toolkit activated, you can draw a curved 3D surface to split a segment into two along this interface.
+
+## Workflow
+
+The recommended steps for splitting a segment are as follows:
+
+1. Use the bounding box tool to create a box around the merge error you want to correct. Theoretically, this step is optional, but keep in mind that relabeling large parts of a segment requires a lot of computing resources and is limited by your computer performance. Consider using the [proofreading tool](../proofreading/tools.html) for large-scale corrections. If you want to correct data for the purpose of ground-truth, it is often enough to only do a local correction instead of proofreading the whole dataset.
+2. Go to the first z slice of the bounding box.
+3. Create a new skeleton tree and place nodes on the boundary that should split the segment into two. Consider activating the "Pen" mode within the skeleton tool so that you can rapidly draw nodes by dragging the mouse while it's pressed down.
+4. Go to the last z slice of the bounding box and repeat the previous step.
+5. Now you can inspect all the slices in between. A smooth spline curve will be shown in each slice that is derived from interpolating between adjacent slices. If you want to correct the interpolation, you can create new nodes on a slice. A new spline curve will be computed from these points and the interpolation will be updated.
+6. If you are satisfied with the (interpolated) boundary, you can switch to the fill tool and relabel one side, effectively splitting a segment into two. Ensure to enable the 3D mode as well as the "Restrict to Bounding Box" mode.
+
+## Troubleshooting
+
+- It can easily happen that the flood-fill operation relabels the entire segment instead of only the part you intended to relabel. This is usually caused by the 3D surface not completely cutting through the segment. Instead, the surface might only go very close to the edge. In that case, the flood-fill operation can traverse over that edge and thus relabel everything (see image below). If this happens, you should undo the fill operation and double-check the surface. To avoid this problem, it is recommended to draw the splitting surface a bit bigger than the actual segment boundaries.
+- If multiple spline curves appear per slice, this is often due to a configuration issue. Please adapt the clipping distance in the left sidebar (`Layers` → `Skeleton`).
+
+![A visualization of a floodfill operation "bleeding" across the boundary because the boundary is not precise enough](../images/splitting-floodfill-visualization.png)
+
+## Impact of "Split Segments" Toolkit
+
+Note that the workflow above is only possible because the "Split Segments" toolkit is enabled.
+By activating that toolkit, WEBKNOSSOS behaves differently in the following way:
+
+- The active tree will be rendered without any edges. Instead, for each slice in which multiple nodes exist, a spline curve is calculated that goes through these nodes.
+- Using the spline curves from before, a 3D surface is generated that interpolates through these curves. The surface can be seen in the 3D viewport.
+- The flood-fill tool will respect the splitting surface by not crossing it.

docs/skeleton_annotation/modes.md

@@ -1,6 +1,6 @@
-# Annotation Modes
+# View Modes
 
-WEBKNOSSOS supports several modes for displaying your dataset & interacting with skeleton annotations.
+WEBKNOSSOS supports several modes for displaying your dataset & interacting with (skeleton) annotations.
 
 ## Orthogonal Mode
 

docs/skeleton_annotation/tools.md

@@ -59,6 +59,14 @@ The following additional modes are available for the skeleton tool beyond the ba
 - Useful for correcting over-segmentation
 - Learn more about [Merger Mode](../proofreading/merger_mode.md)
 
+
+![Skeleton Pen Mode](./images/pen-mode-modifier.jpg){align=left width="60"} 
+**Skeleton Pen Mode**
+
+- If this mode is enabled, the skeleton tool behaves similarly to the brush tool: holding down the left mouse button and dragging the mouse will continuously create nodes.
+- Selecting or moving nodes via drag-and-drop is disabled in this mode.
+
+
 ## Skeleton Keyboard Shortcuts 
 
 The following common keyboard shortcuts are handy for speeding up your annotation workflow:
@@ -76,4 +84,4 @@ The following common keyboard shortcuts are handy for speeding up your annotatio
 | ++c++           | Create New Tree                                    |
 
 !!! tip "Keyboard Shortcuts"
-    For faster workflow, refer to the [keyboard shortcuts](../ui/keyboard_shortcuts.md) guide.
+    For faster workflows, refer to the [keyboard shortcuts](../ui/keyboard_shortcuts.md) guide.

docs/ui/index.md

@@ -2,7 +2,7 @@
 
 The main WEBKNOSSOS user interface for viewing and annotating datasets is divided into five sections:
 
-1. A [toolbar](./toolbar.md) for general purposes features such as Saving your work and displaying the current position within the dataset. Further, it provides access to all the tools for annotating and manipulating your data. It spans along the full width of the top of your screen.
+1. A [toolbar](./toolbar.md) for general-purpose features such as saving your work and displaying the current position within the dataset. Further, it provides access to all the tools for annotating and manipulating your data. It spans along the full width of the top of your screen.
 2. The [left-hand side panel](./layers.md) provides a list of all available data, segmentation, and annotation layers as well as a settings menu for viewport options and keyboard controls.
 3. The center of the screen is occupied by the annotation interface. Your dataset is displayed here, and you navigate and annotate it as desired. Most interactions will take place here.
 4. The [right-hand side panel](./object_info.md) is occupied by several tabs providing more information on your current dataset, skeleton/volume annotations, and other lists. Depending on your editing mode these tabs might adapt.

docs/ui/toolbar.md

@@ -58,6 +58,29 @@ WEBKNOSSOS offers multiple ways to share your work:
 
 - **Layout**: Modify the layout of the WEBKNOSSOS user interface and resize, reorder and adjust viewports and panels to your preferences. Customize the number of columns, show or hide specific tabs, and adjust the size of the sidebar. Save and restore your preferred configurations. 
 
+## Modes and Toolkits
+
+### View Mode
+
+Datasets can be viewed in different view modes.
+Read more about these [here](../skeleton_annotation/modes.md)
+
+### Toolkits
+
+WEBKNOSSOS offers several tools that you can use to interact with your datasets.
+By default, all tools are available, but it is also possible to only show a subset of tools.
+These tool subsets are called "toolkits".
+Some of these also fine-tune the behavior of the tools so that they are tailored towards a specific use case.
+
+Currently, there are four toolkits available:
+
+- All Tools: This toolkit contains all available tools and is the default.
+- Read Only: Only tools that cannot mutate the annotation are available (i.e., move and measuring tools).
+- Volume: Only volume tools are listed here.
+- Split Segments: This toolkit allows splitting a labeled segment into two parts. Tools in this toolkit behave a bit differently to streamline this workflow. Read more about this toolkit [here](../proofreading/split_segments_toolkit.md).
+
+![Toolkit Selection](../images/toolkit_dropdown.jpg)
+
 ## Annotation Tools
 
 ### Navigation

frontend/javascripts/libs/UpdatableTexture.ts

@@ -33,10 +33,9 @@ class UpdatableTexture extends THREE.Texture {
     mapping?: THREE.Mapping,
     wrapS?: THREE.Wrapping,
     wrapT?: THREE.Wrapping,
-    magFilter?: THREE.TextureFilter,
-    minFilter?: THREE.TextureFilter,
+    magFilter?: THREE.MagnificationTextureFilter,
+    minFilter?: THREE.MinificationTextureFilter,
     anisotropy?: number,
-    encoding?: THREE.TextureEncoding,
   ) {
     const imageData = { width, height, data: new Uint32Array(0) };
 
@@ -51,7 +50,6 @@ class UpdatableTexture extends THREE.Texture {
       format,
       type,
       anisotropy,
-      encoding,
     );
 
     this.magFilter = magFilter !== undefined ? magFilter : THREE.LinearFilter;
@@ -65,15 +63,11 @@ class UpdatableTexture extends THREE.Texture {
   setRenderer(renderer: THREE.WebGLRenderer) {
     this.renderer = renderer;
     this.gl = this.renderer.getContext() as WebGL2RenderingContext;
-    this.utils = new THREE.WebGLUtils(
-      this.gl,
-      this.renderer.extensions,
-      this.renderer.capabilities,
-    );
+    this.utils = new THREE.WebGLUtils(this.gl, this.renderer.extensions);
   }
 
   isInitialized() {
-    return this.renderer.properties.get(this).__webglTexture != null;
+    return (this.renderer.properties.get(this) as any).__webglTexture != null;
   }
 
   update(src: TypedArray, x: number, y: number, width: number, height: number) {
@@ -94,7 +88,7 @@ class UpdatableTexture extends THREE.Texture {
       this.renderer.initTexture(this);
     }
     const activeTexture = this.gl.getParameter(this.gl.TEXTURE_BINDING_2D);
-    const textureProperties = this.renderer.properties.get(this);
+    const textureProperties = this.renderer.properties.get(this) as any;
     this.gl.bindTexture(this.gl.TEXTURE_2D, textureProperties.__webglTexture);
 
     originalTexSubImage2D(

frontend/javascripts/libs/compute_bvh_async.ts

@@ -1,3 +1,4 @@
+import type * as THREE from "three";
 import type { MeshBVH } from "three-mesh-bvh";
 // @ts-ignore
 import { GenerateMeshBVHWorker } from "three-mesh-bvh/src/workers/GenerateMeshBVHWorker";

frontend/javascripts/libs/cuckoo/abstract_cuckoo_table.ts

@@ -32,7 +32,7 @@ export abstract class AbstractCuckooTable<K, V, Entry extends [K, V]> {
     return THREE.UnsignedIntType;
   }
 
-  static getTextureFormat() {
+  static getTextureFormat(): THREE.PixelFormat {
     return THREE.RGBAIntegerFormat;
   }
 

frontend/javascripts/libs/order_points_with_mst.ts

@@ -0,0 +1,119 @@
+import type * as THREE from "three";
+
+export function orderPointsWithMST(points: THREE.Vector3[]): THREE.Vector3[] {
+  /*
+   * Find the order of points with the shortest distance heuristically.
+   * This is done by computing the MST of the points and then traversing
+   * that MST several times (each node is tried as the starting point).
+   * The shortest order wins.
+   */
+  if (points.length === 0) return [];
+
+  const mst = computeMST(points);
+  let bestOrder: number[] = [];
+  let minLength = Number.POSITIVE_INFINITY;
+
+  for (let startIdx = 0; startIdx < points.length; startIdx++) {
+    const order = traverseMstDfs(mst, startIdx);
+    const length = computePathLength(points, order);
+
+    if (length < minLength) {
+      minLength = length;
+      bestOrder = order;
+    }
+  }
+
+  return bestOrder.map((index) => points[index]);
+}
+
+// Mostly generated with ChatGPT:
+
+class DisjointSet {
+  // Union find datastructure
+  private parent: number[];
+  private rank: number[];
+
+  constructor(n: number) {
+    this.parent = Array.from({ length: n }, (_, i) => i);
+    this.rank = Array(n).fill(0);
+  }
+
+  find(i: number): number {
+    if (this.parent[i] !== i) this.parent[i] = this.find(this.parent[i]);
+    return this.parent[i];
+  }
+
+  union(i: number, j: number): void {
+    let rootI = this.find(i),
+      rootJ = this.find(j);
+    if (rootI !== rootJ) {
+      if (this.rank[rootI] > this.rank[rootJ]) this.parent[rootJ] = rootI;
+      else if (this.rank[rootI] < this.rank[rootJ]) this.parent[rootI] = rootJ;
+      else {
+        this.parent[rootJ] = rootI;
+        this.rank[rootI]++;
+      }
+    }
+  }
+}
+
+interface Edge {
+  i: number;
+  j: number;
+  dist: number;
+}
+
+function computeMST(points: THREE.Vector3[]): number[][] {
+  const edges: Edge[] = [];
+  const numPoints = points.length;
+
+  // Create all possible edges with distances
+  for (let i = 0; i < numPoints; i++) {
+    for (let j = i + 1; j < numPoints; j++) {
+      const dist = points[i].distanceTo(points[j]);
+      edges.push({ i, j, dist });
+    }
+  }
+
+  // Sort edges by distance (Kruskal's Algorithm)
+  edges.sort((a, b) => a.dist - b.dist);
+
+  // Compute MST using Kruskal’s Algorithm
+  const ds = new DisjointSet(numPoints);
+  const mst: number[][] = Array.from({ length: numPoints }, () => []);
+
+  for (const { i, j } of edges) {
+    if (ds.find(i) !== ds.find(j)) {
+      ds.union(i, j);
+      mst[i].push(j);
+      mst[j].push(i);
+    }
+  }
+
+  return mst;
+}
+
+function traverseMstDfs(mst: number[][], startIdx = 0): number[] {
+  const visited = new Set<number>();
+  const orderedPoints: number[] = [];
+
+  function dfs(node: number) {
+    if (visited.has(node)) return;
+    visited.add(node);
+    orderedPoints.push(node);
+    for (let neighbor of mst[node]) {
+      dfs(neighbor);
+    }
+  }
+
+  dfs(startIdx);
+  return orderedPoints;
+}
+
+function computePathLength(points: THREE.Vector3[], order: number[]): number {
+  let length = 0;
+  for (let i = 0; i < order.length - 1; i++) {
+    length += points[order[i]].distanceTo(points[order[i + 1]]);
+  }
+  return length;
+}

frontend/javascripts/libs/window.ts

@@ -1,6 +1,7 @@
 // This module should be used to access the window object, so it can be mocked in the unit tests
 
 import type TextureBucketManager from "oxalis/model/bucket_data_handling/texture_bucket_manager";
+import type * as THREE from "three";
 import type { ArbitraryFunction, ArbitraryObject } from "types/globals";
 
 const removeEventListener = (

frontend/javascripts/messages.tsx

@@ -477,7 +477,7 @@ instead. Only enable this option if you understand its effect. All layers will n
     "<%- userName %> is about to become a dataset manager and will be able to access and edit all datasets within this organization.",
   ),
   "users.set_admin": _.template(
-    "<%- userName %> is about to become an admin for this organization with full read/write access to all datasets and management capbilities for all users, projects, and tasks.",
+    "<%- userName %> is about to become an admin for this organization with full read/write access to all datasets and management capabilities for all users, projects, and tasks.",
   ),
   "users.change_email_title": "Do you really want to change the email?",
   "users.change_email": _.template(