Skip to content

Support "binary vectors": @JsonFormat(shape = Shape.BINARY) for float[], double[] #5242

@cowtowncoder

Description

@cowtowncoder

The default method of serializing float[] and double[] is as regular JSON Array with floating-point numbers. But as per:

https://medium.com/@cowtowncoder/super-fast-packed-base64-vectors-dc88cf5253ff

there is a usually MUCH faster-to-process (... and usually more compact too!) representation where in serialization is done by:

  • float / double values are converted to their raw binary 32- / 64-bit representations
  • these representations are marshalled in Network Byte Order into (logical) byte[]
  • that logical byte[] array is Base64-encoded as a JSON String for JSON (and native Binary for formats that support that).

Deserialization then proceeds in the reverse direction.

Configuration for serialization should use @JsonFormat with the usual multi-level configuration:

  1. Global configOverrides allow specifying this for all float[] and/or double[] valued properties
  2. Per-property @JsonFormat allows choice for specific property.

Metadata

Metadata

Assignees

Labels

2.20Issues planned at 2.20 or later

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions