Skip to content

In Data Loading, Clip to Layer BoundingBox #8551

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Apr 28, 2025
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.unreleased.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ For upgrade instructions, please check the [migration guide](MIGRATIONS.released
- When uploading multiple NMLs at once, the description is now kept, if all NMLs with non-empty descriptions have the same description. [#8533](https://github.com/scalableminds/webknossos/pull/8533)

### Changed
- When loading data from a data layer that has data stored beyond the bounding box specified in the datasource-properties.json, data outside of the bounding box is now zeroed. (the layer is “clipped”). [#8551](https://github.com/scalableminds/webknossos/pull/8551)

### Fixed

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import play.api.libs.json.{JsObject, Json}

case class BoundingBox(topLeft: Vec3Int, width: Int, height: Int, depth: Int) {

val bottomRight: Vec3Int = topLeft.move(width, height, depth)
lazy val bottomRight: Vec3Int = topLeft.move(width, height, depth)

def intersects(other: BoundingBox): Boolean =
math.max(topLeft.x, other.topLeft.x) < math.min(bottomRight.x, other.bottomRight.x) &&
Expand Down Expand Up @@ -45,6 +45,9 @@ case class BoundingBox(topLeft: Vec3Int, width: Int, height: Int, depth: Int) {
BoundingBox(Vec3Int(x, y, z), w, h, d)
}

def isFullyContainedIn(other: BoundingBox): Boolean =
this.intersection(other).contains(this)

def isEmpty: Boolean =
width <= 0 || height <= 0 || depth <= 0

Expand All @@ -61,6 +64,9 @@ case class BoundingBox(topLeft: Vec3Int, width: Int, height: Int, depth: Int) {
// Since floorDiv is used for topLeft, ceilDiv is used for the size to avoid voxels being lost at the border
BoundingBox(topLeft / that, ceilDiv(width, that.x), ceilDiv(height, that.y), ceilDiv(depth, that.z))

def move(delta: Vec3Int): BoundingBox =
this.copy(topLeft = this.topLeft + delta)

def toSql: List[Int] =
List(topLeft.x, topLeft.y, topLeft.z, width, height, depth)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ case class Vec3Int(x: Int, y: Int, z: Int) {
def /(that: Vec3Int): Vec3Int =
Vec3Int(x / that.x, y / that.y, z / that.z)

def unary_- : Vec3Int =
Vec3Int(-x, -y, -z)

def scale(s: Float): Vec3Int =
Vec3Int((x * s).toInt, (y * s).toInt, (z * s).toInt)

Expand Down Expand Up @@ -53,8 +56,6 @@ case class Vec3Int(x: Int, y: Int, z: Int) {
def move(other: Vec3Int): Vec3Int =
move(other.x, other.y, other.z)

def negate: Vec3Int = Vec3Int(-x, -y, -z)

def to(bottomRight: Vec3Int): Seq[Vec3Int] =
range(bottomRight, _ to _)

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
package com.scalableminds.webknossos.datastore.models.requests

import com.scalableminds.util.geometry.Vec3Int
import com.scalableminds.util.geometry.{BoundingBox, Vec3Int}
import com.scalableminds.webknossos.datastore.models.{BucketPosition, VoxelPosition}

/**
Expand Down Expand Up @@ -51,4 +51,10 @@ case class Cuboid(topLeft: VoxelPosition, width: Int, height: Int, depth: Int) {
height * mag.y,
depth * mag.z
)

def toBoundingBoxInMag: BoundingBox =
BoundingBox(Vec3Int(topLeft.voxelXInMag, topLeft.voxelYInMag, topLeft.voxelZInMag), width, height, depth)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't width, height, depth also be in mag?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they already are, they are just not named as explicitly


def toMag1BoundingBox: BoundingBox =
toMag1.toBoundingBoxInMag
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package com.scalableminds.webknossos.datastore.models.requests

import com.scalableminds.util.geometry.Vec3Int
import com.scalableminds.webknossos.datastore.models.{AdditionalCoordinate, BucketPosition}
import com.scalableminds.webknossos.datastore.models.datasource.{DataLayer, DataSource, SegmentationLayer}

Expand All @@ -21,6 +22,7 @@ case class DataServiceDataRequest(
settings: DataServiceRequestSettings
) {
def isSingleBucket: Boolean = cuboid.isSingleBucket(DataLayer.bucketLength)
def mag: Vec3Int = cuboid.mag
}

case class DataReadInstruction(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -106,16 +106,51 @@ class BinaryDataService(val dataBaseDir: Path,
conversionFunc(inputArray))
else Full(inputArray)

/*
* Everything outside of the layer bounding box is set to black (zero) so data outside of the specified
* bounding box is not exposed to the user
*/
private def clipToLayerBoundingBox(request: DataServiceDataRequest)(inputArray: Array[Byte]): Box[Array[Byte]] = {
val bytesPerElement = request.dataLayer.bytesPerElement
val requestBboxInMag = request.cuboid.toBoundingBoxInMag
val layerBboxInMag = request.dataLayer.boundingBox / request.mag // Note that this div is implemented to round to the bigger bbox so we don’t lose voxels inside.
val intersectionOpt = requestBboxInMag.intersection(layerBboxInMag).map(_.move(-requestBboxInMag.topLeft))
val outputArray = Array.fill[Byte](inputArray.length)(0)
intersectionOpt.foreach { intersection =>
for {
z <- intersection.topLeft.z until intersection.bottomRight.z
y <- intersection.topLeft.y until intersection.bottomRight.y
// We can bulk copy a row of voxels and do not need to iterate in the x dimension
} {
val offset =
(intersection.topLeft.x +
y * requestBboxInMag.width +
z * requestBboxInMag.width * requestBboxInMag.height) * bytesPerElement
System.arraycopy(inputArray,
offset,
outputArray,
offset,
(intersection.bottomRight.x - intersection.topLeft.x) * bytesPerElement)
Comment on lines +131 to +135
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this is the most likely cause for the nullpointer

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some out of bounds maybe. Maybe even due to my comment above regarding them dimensions not being converted to the respective mag

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the record, the null was explicitly set for request.dataSource, compare #8573 (comment)

}
}
Full(outputArray)
}

private def convertAccordingToRequest(request: DataServiceDataRequest, inputArray: Array[Byte]): Fox[Array[Byte]] =
for {
clippedData <- convertIfNecessary(
!request.cuboid.toMag1BoundingBox.isFullyContainedIn(request.dataLayer.boundingBox),
inputArray,
clipToLayerBoundingBox(request),
request)
mappedDataFox <- agglomerateServiceOpt.map { agglomerateService =>
convertIfNecessary(
request.settings.appliedAgglomerate.isDefined && request.dataLayer.category == Category.segmentation && request.cuboid.mag.maxDim <= MaxMagForAgglomerateMapping,
inputArray,
clippedData,
agglomerateService.applyAgglomerate(request),
request
)
}.fillEmpty(Fox.successful(inputArray)) ?~> "Failed to apply agglomerate mapping"
}.fillEmpty(Fox.successful(clippedData)) ?~> "Failed to apply agglomerate mapping"
mappedData <- mappedDataFox
resultData <- convertIfNecessary(request.settings.halfByte, mappedData, convertToHalfByte, request)
} yield resultData
Expand Down Expand Up @@ -164,7 +199,6 @@ class BinaryDataService(val dataBaseDir: Path,
private def cutOutCuboid(request: DataServiceDataRequest, rs: List[(BucketPosition, Array[Byte])]): Array[Byte] = {
val bytesPerElement = request.dataLayer.bytesPerElement
val cuboid = request.cuboid
val subsamplingStrides = Vec3Int.ones

val resultShape = Vec3Int(cuboid.width, cuboid.height, cuboid.depth)
val result = new Array[Byte](cuboid.volume * bytesPerElement)
Expand All @@ -190,9 +224,9 @@ class BinaryDataService(val dataBaseDir: Path,
y % bucketLength * bucketLength +
z % bucketLength * bucketLength * bucketLength) * bytesPerElement

val rx = (xMin - cuboid.topLeft.voxelXInMag) / subsamplingStrides.x
val ry = (y - cuboid.topLeft.voxelYInMag) / subsamplingStrides.y
val rz = (z - cuboid.topLeft.voxelZInMag) / subsamplingStrides.z
val rx = xMin - cuboid.topLeft.voxelXInMag
val ry = y - cuboid.topLeft.voxelYInMag
val rz = z - cuboid.topLeft.voxelZInMag

val resultOffset = (rx + ry * resultShape.x + rz * resultShape.x * resultShape.y) * bytesPerElement
System.arraycopy(data, dataOffset, result, resultOffset, (xMax - xMin) * bytesPerElement)
Expand Down