Skip to content

Performance improvements and Refactorings for FossilDbIterators #8211

@fm3

Description

@fm3

VersionedFossilDbIterator and its dependents are used when multiple keys of fossildb-saved data are processed sequentially. Some ideas how to improve them:

  • Cleanup: Generic Reversion-Aware Iterator? Build on top of VersionedFossilDbIterator? Could replace VersionedAgglomerateToGraphIterator, VersionedSegmentToAgglomerateChunkIterator and VolumeTracingBucketProvider?
  • For proofreading iterators, save perf by skipping fetching content when only version is needed in iterator (different from volume, because here we don’t need to update the segment index)
    • that might need new fossildb api (ListKeysWithVersions)
  • Make the iterators async? Use async fossildb api?
  • Can we get rid of the Option.get in next()?

Should be tackled after #7839 is merged

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions