Make user bounding box updates more efficient #8492

MichaelBuessemeyer · 2025-04-01T16:49:04Z

This PR add more update actions for user bounding boxes. This makes it more efficient to update a single bounding box (including adding and deleting). Moreover, the version restore view can display more details on what was actually changed (e.g. color of the bounding box).

URL of deployed dev instance (used for testing):

https://___.webknossos.xyz

Steps to test:

abc

TODOs:

if code is kept: make saga code DRY in volumetracing_saga and skeletontracing_saga
use updateUserBoundingBoxVisibility actions for visibility updates
- frontend
- backend
consider renaming actions (no "action" in name field AND new action name, e.g. updateUserSpecificBoundingBox[Volume|Skeleton])
- frontend (done: no action in name field)
- backend

Issues:

fixes #7604

(Please delete unneeded items, merge only when none are left open)

Updated changelog
Updated migration guide if applicable
Updated documentation if applicable
Adapted wk-libs python client if relevant API parts change
Removed dev-only changes like prints and application.conf edits
Considered common edge cases
Needs datastore update after deployment

coderabbitai · 2025-04-01T16:49:11Z

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Generate unit testing code for this file.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit testing code for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read src/utils.ts and generate unit testing code.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai generate docstrings to generate docstrings for this PR.
@coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

fm3 · 2025-04-01T16:55:42Z

@MichaelBuessemeyer How would you feel about three new actions, Add/Delete/Update bbox? That way we don’t need an update action for every property.
It’s true that the version restore view will be a little less detailed then, but I’d say that’s a fair price for the reduced code complexity. Maybe @philippotto also has an opinion? TBH I think there’s already an overwhelming amount of update action classes…

MichaelBuessemeyer · 2025-04-03T07:26:02Z

@MichaelBuessemeyer How would you feel about three new actions, Add/Delete/Update bbox? That way we don’t need an update action for every property. It’s true that the version restore view will be a little less detailed then, but I’d say that’s a fair price for the reduced code complexity. Maybe @philippotto also has an opinion? TBH I think there’s already an overwhelming amount of update action classes…

Yes, I agree that there are already a lot of update actions. My idea was to follow how it is done for skeleton update actions. There we also have the update action on such a detailed degree. Moreover, when thinking about the upcoming collab feature, the design doc states:

update actions of segment metadata (and maybe trees, too, for consistency) should be updated per key (so that setting two different keys does not overwrite each other)

I can image the same problem with bounding boxes:

User A renames BBox with Id x
User B re-colors the BBox with Id x
User A saves
User B receives update:

At 4. the frontend now needs to be really smart to first find out by a diff that the name of the bounding box was changed. A simple comparison with the current BBox state of User B is not possible as the diff would also reveal that the color changed (done by User B). But only the name did and only the name should be updated.

This complexity could be saved in case we have an action for each field making things easier implementation wise (at least that's how I see it on first thought 🤔)

Although as the bounding boxes state are saved at the time Users B last successful update was submitted to the backend, there should be a state that can be compared with to find out that actually only the name changed. But I am not sure how complicated such a code would look like compared to a "updateBBoxNameAction" and simply applying it 🤔.

To me more update actions sound more tedious code-wise but should be more straight forward to implement leading to more code lines but less code complexity

philippotto · 2025-04-03T07:40:10Z

I'm also in favor of having less update action types. I think we should aim for update actions where not all properties have to be listed. For example, one could have UpdateBoundingBox(name="new name") and UpdateBoundingBox(color="new color"). then, we only have one class for the update, but the sets of changed keys can be different.

I think, we already had an issue for this, but I cannot find it anymore. from what I remember, the above is a bit tricky, because scala cannot distinguish between undefined and simply not listed in a JSON? or something like this? however, we should be able to work around this somehow (e.g., list the keys explicitly that should be set).

fm3 · 2025-04-03T09:45:26Z

You are probably referring to #6657 – there we didn’t consider this for update actions but it might just work there too.

… bounding boxes

knollengewaechs · 2025-04-03T17:32:56Z

what timestamp should be put into the updateAction? the moment the first frontend action was fired or the moment when the annotation is saved?

edit: this is done within update_actions.ts with the types AddServerValues, AsServerAction and ServerUpdateAction. No need to add this to the new actions explicitly.

knollengewaechs · 2025-04-03T18:15:57Z

however, we should be able to work around this somehow (e.g., list the keys explicitly that should be set).

my initial solution is to call the frontend updateAction with a Partial<UserBoundindBox> and when sending this to the backend, explicitly include the props in the partial as a list. if anyone thinks that this is a bad idea, please let me know!

edit: talked with @MichaelBuessemeyer, seems to be fine

…some places

knollengewaechs · 2025-04-04T13:09:49Z

@MichaelBuessemeyer I updated the frontend code, so that new the new updateActions are called. I tested it a bit with the old backend code, and it seems like the right actions are dispatched if the bounding boxes change. Let me know if the code seems buggy in any way!
And as you can tell, I already added some actions for VolumeTracings aswell.

frontend/javascripts/oxalis/model/sagas/skeletontracing_saga.ts

…e-efficient-bbox-update-actions

…w add update actions

MichaelBuessemeyer · 2025-04-08T17:04:34Z

@MichaelBuessemeyer I updated the frontend code, so that new the new updateActions are called. I tested it a bit with the old backend code, and it seems like the right actions are dispatched if the bounding boxes change.

Nice 👍 awesome. Thanks a lot 🙏
I changed the frontend code a little to not return single elemented lists of user bounding boxes for the add actions. See c680cf2. This was necessary for further testing.

Let me know if the code seems buggy in any way!

Yeah sorry there is a little buggy / unexpected behaviour:
When the bounds of a bbox are changed the changed bounds are not send as a bounding box object with topLeft, width, height, depth but as changes in the min max props. See the screenshot below (taken from the network tab showing a update actions transaction dispatched to the server):

Could you please adjust the frontend to always send the whole bounds as an object (topLeft, width, height, depth) in case some of them were updated? Thanks a lot 🙏

Moreover, I noticed that the frontend seems to send "empty" updates: The updatedPropKeys and updatedProps are empty. This is also shown in the screenshot in the first array entries value. Could you please investigate this and try to avoid such empty update?

Moreover, the cherry on top would be that multiple consecutive bounding box updates to the same bounding box would be combined into a single action. Something like this is already done by the frontend before sending the update actions. See compact_save_queue.ts ll. 100ff. But I am unaware how easy this is to implement. So maybe that's a follow up 🤔?

knollengewaechs · 2025-04-09T11:19:21Z

@MichaelBuessemeyer thanks for the reports! I will work on this now and try to fix the three problems.

knollengewaechs · 2025-04-10T11:25:33Z

okay, I adjusted the frontend code so that

the whole bounds object is sent
empty updates are omitted
consecutive updates on the same bbox on the same prop are out into one action. this is especially important for bounds updates when the user uses the dragging option. One problem is that often multiple actions are put into one batch, and then they cant be omitted. at least thats how its done for other types of actions, e.g. when updating segment.
let me know if you have more suggestions for this!

knollengewaechs · 2025-04-10T11:33:52Z

let me know if you notice anything else. during my testing, adding, updating and deleting bboxes seems to work fine :)

MichaelBuessemeyer · 2025-04-14T08:22:07Z

For reference: For now blocked. See https://scm.slack.com/archives/C5AKLAV0B/p1744295336264969

knollengewaechs · 2025-04-23T10:32:02Z

...os-tracingstore/app/com/scalableminds/webknossos/tracingstore/annotation/UpdateActions.scala

          case "updateUserBoundingBoxesInVolumeTracing" =>
            deserialize[UpdateUserBoundingBoxesVolumeAction](jsonValue)
-          case "updateUserBoundingBoxVisibilityInVolumeTracing" =>


@MichaelBuessemeyer fyi I removed the "in" in the string -> the new old actions have the names "updateUserBoundingBoxVisibilitySkeletonAction" and "updateUserBoundingBoxVisibilityVolumeAction"

Take care not to mix the scala class names vs the "name" field (string above). This is different for most actions. I think the "name" usually does not end on "Action"

yes, I was talking about the "name" field. maybe we have to update the names of the actions that are introduced with this PR because they all end on "Action" 😅

I took the freedom to change the actions names because I think it was me who might have introduced this by mistake. @MichaelBuessemeyer feel free to revert this (675ae35) if you disagree

knollengewaechs · 2025-04-25T15:46:00Z

frontend/javascripts/oxalis/model/sagas/update_actions.ts

@@ -38,11 +41,29 @@ export type CreateSegmentUpdateAction = ReturnType<typeof createSegmentVolumeAct
 export type UpdateSegmentUpdateAction = ReturnType<typeof updateSegmentVolumeAction>;
 export type DeleteSegmentUpdateAction = ReturnType<typeof deleteSegmentVolumeAction>;
 export type DeleteSegmentDataUpdateAction = ReturnType<typeof deleteSegmentDataVolumeAction>;
-type UpdateUserBoundingBoxesInSkeletonTracingUpdateAction = ReturnType<
-  typeof updateUserBoundingBoxesInSkeletonTracing
+export type AddUserBoundingBoxSkeletonAction = ReturnType<


add more supported update actions for user bounding box updates

7bcb4c7

MichaelBuessemeyer added backend frontend labels Apr 1, 2025

MichaelBuessemeyer self-assigned this Apr 1, 2025

MichaelBuessemeyer mentioned this pull request Apr 1, 2025

More efficiently update User Bounding Boxes #7604

Open

[frontend] add new updateActions to add, remove and update individual…

09de156

… bounding boxes

knollengewaechs added 3 commits April 3, 2025 20:45

WIP: adjust saga code to use new update actions and fixing typing in …

7a187fd

…some places

[frontend] remove activeUser, timestamp and info from updateActions

cdf546f

[frontend] make saga code more DRY and only update changed props

bfb8746

knollengewaechs self-assigned this Apr 4, 2025

[frontend] correctly call helper function

78d8d18

knollengewaechs reviewed Apr 4, 2025

View reviewed changes

frontend/javascripts/oxalis/model/sagas/skeletontracing_saga.ts Outdated Show resolved Hide resolved

MichaelBuessemeyer added 4 commits April 7, 2025 14:35

Merge branch 'master' of github.com:scalableminds/webknossos into mor…

6f20124

…e-efficient-bbox-update-actions

Merge branch 'master' of github.com:scalableminds/webknossos into mor…

01323e2

…e-efficient-bbox-update-actions

have single update action for all user bounding box properties

c680cf2

Don't make frontend return a single elemnt list of a user bbox for ne…

55fdf45

…w add update actions

knollengewaechs added 4 commits April 9, 2025 14:48

[frontend] dont send empty update objects

d8dcfdd

[frontend] WIP: send right bbox object to server

b8d0a71

[frontend] clean up code

21e13e2

[frontend] remove unneccesary update actions sent to backend

374fe71

[frontend] remove old actions

c6a9b0f

knollengewaechs reviewed Apr 23, 2025

View reviewed changes

knollengewaechs added 2 commits April 23, 2025 14:19

[frontend] use updateUserBoundingBoxVisibilityActions

0ae286c

[frontend] adjust new update actions name fields

675ae35

knollengewaechs reviewed Apr 25, 2025

View reviewed changes

[frontend] rename actions to ...In[Volume|Skeleton]TracingAction

4d33b5c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make user bounding box updates more efficient #8492

Make user bounding box updates more efficient #8492

MichaelBuessemeyer commented Apr 1, 2025 •

edited by knollengewaechs

Loading

coderabbitai bot commented Apr 1, 2025 •

edited

Loading

Review skipped

Chat

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

CodeRabbit Configuration File (`.coderabbit.yaml`)

Documentation and Community

fm3 commented Apr 1, 2025 •

edited

Loading

MichaelBuessemeyer commented Apr 3, 2025

philippotto commented Apr 3, 2025

fm3 commented Apr 3, 2025

knollengewaechs commented Apr 3, 2025 •

edited

Loading

knollengewaechs commented Apr 3, 2025 •

edited

Loading

knollengewaechs commented Apr 4, 2025 •

edited

Loading

MichaelBuessemeyer commented Apr 8, 2025

knollengewaechs commented Apr 9, 2025

knollengewaechs commented Apr 10, 2025 •

edited

Loading

knollengewaechs commented Apr 10, 2025

MichaelBuessemeyer commented Apr 14, 2025

knollengewaechs Apr 23, 2025

fm3 Apr 23, 2025

knollengewaechs Apr 23, 2025

knollengewaechs Apr 23, 2025 •

edited

Loading

knollengewaechs Apr 25, 2025

Make user bounding box updates more efficient #8492

Are you sure you want to change the base?

Make user bounding box updates more efficient #8492

Conversation

MichaelBuessemeyer commented Apr 1, 2025 • edited by knollengewaechs Loading

URL of deployed dev instance (used for testing):

Steps to test:

TODOs:

Issues:

coderabbitai bot commented Apr 1, 2025 • edited Loading

Review skipped

Chat

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

CodeRabbit Configuration File (.coderabbit.yaml)

Documentation and Community

fm3 commented Apr 1, 2025 • edited Loading

MichaelBuessemeyer commented Apr 3, 2025

philippotto commented Apr 3, 2025

fm3 commented Apr 3, 2025

knollengewaechs commented Apr 3, 2025 • edited Loading

knollengewaechs commented Apr 3, 2025 • edited Loading

knollengewaechs commented Apr 4, 2025 • edited Loading

MichaelBuessemeyer commented Apr 8, 2025

knollengewaechs commented Apr 9, 2025

knollengewaechs commented Apr 10, 2025 • edited Loading

knollengewaechs commented Apr 10, 2025

MichaelBuessemeyer commented Apr 14, 2025

knollengewaechs Apr 23, 2025

Choose a reason for hiding this comment

fm3 Apr 23, 2025

Choose a reason for hiding this comment

knollengewaechs Apr 23, 2025

Choose a reason for hiding this comment

knollengewaechs Apr 23, 2025 • edited Loading

Choose a reason for hiding this comment

knollengewaechs Apr 25, 2025

Choose a reason for hiding this comment

MichaelBuessemeyer commented Apr 1, 2025 •

edited by knollengewaechs

Loading

coderabbitai bot commented Apr 1, 2025 •

edited

Loading

CodeRabbit Configuration File (`.coderabbit.yaml`)

fm3 commented Apr 1, 2025 •

edited

Loading

knollengewaechs commented Apr 3, 2025 •

edited

Loading

knollengewaechs commented Apr 3, 2025 •

edited

Loading

knollengewaechs commented Apr 4, 2025 •

edited

Loading

knollengewaechs commented Apr 10, 2025 •

edited

Loading

knollengewaechs Apr 23, 2025 •

edited

Loading