Enable alignment of sessions in a common coordinate system via RoI-based transforms

[niksirbi]

The Problem

It's common to acquire multiple videos or sessions of the same setup. Ideally, if both the setup and the camera remain perfectly stationary, we could analyse data across sessions using a shared coordinate system. However, in practice, the camera and/or the setup often shift between sessions, and there may be changes in field of view, angle, etc.

Proposed Solution

We can leverage the RegionOfInterest class to compute transforms between sessions. Here's the idea:

• RoIs (both lines and polygons) are defined by a set of points.
• If we define the same RoI (e.g., the arena) in two videos, we can compute the linear transformation that best aligns those two RoIs.
• This transformation can then be applied to the position data, allowing coordinates tracked in one video to be expressed in the coordinate system of another.

Considerations

RoIs defined across different videos might not have the same number of points—for instance, the arena in video 1 might have 10 vertices, while in video 2 it has 11. To make point matching work reliably, we could uniformly sample a fixed number of points along the polygon's perimeter (or along the line, for linear RoIs). For example, sampling 100 points from each RoI before matching would provide consistency.

Additional Benefits

This approach could also significantly speed up manual RoI drawing. Imagine you need to define 5 RoIs per video: arena, cage, wheel, tube, and water port. Instead of redrawing all 5 in every video, you could:

• Choose one video as the reference and define all 5 RoIs in that space.
• In the remaining videos, define only the arena.
• Register the arena in each video to the reference video’s arena.
• Apply the inverse of the computed transform to map the remaining 4 RoIs from the reference space to each target video.

API Design

I haven’t fully fleshed this out yet, but one approach that fits with our current architecture could be:

• Add an .align() or .register() method to the RegionOfInterest class. This method would accept a reference RoI as input and return a linear transformation matrix.
• Implement an apply_transform() function in movement.transforms. This would take a position DataArray and a linear transformation matrix, and return the transformed position array.

These two functions would provide the core functionality we need. Additional wrappers or syntactic sugar could be added later to improve usability.