-
Notifications
You must be signed in to change notification settings - Fork 297
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Is your feature request related to a problem or challenge?
Currently, iceberg-rust
does not support committing operations that both add and delete files. This feature, known as RewriteFiles
in iceberg-java
, is crucial for our use case, particularly for compaction.
Describe the solution you'd like
Based on my initial investigation in #1606, I believe the following three components are necessary to implement RewriteFiles
in iceberg-rust
:
- Process delete files: We need to add logic to handle delete files when generating snapshots.
- Implement
RewriteFilesAction
: This will require creating a new transaction action,RewriteFilesAction
, and related classes likeRewriteFilesOperation
. - Support validation: We need to add validation for rewrite operations to prevent conflicts. Currently we don't have any validation mechnism in the codebase, so a new trait
SnapshotValidator
would be needed.
Willingness to contribute
I would be willing to contribute to this feature with guidance from the Iceberg Rust community
dentiny
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request