You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The maintenance_tasks engine is an excellent tool for managing data migrations in Rails apps, especially with its built-in support for CSV via csv_collection. However, its current focus on flat, tabular data limits its usability in scenarios where JSON is the preferred format—particularly for data with nested or hierarchical structures.
Problem:
CSV is flat: Difficult to represent nested attributes or complex relationships without workarounds.
Proposal: json_collection
Introduce a json_collection that:
Allows tasks to process uploaded JSON files via Active Storage.
Iterates over each object in a JSON array (or structured path).
Enables custom processing logic per item, similar to csv_collection.
Use Cases
Migrating data from JSON columns in the database.
Handling product catalogs exported as JSON.
Processing arrays of deeply nested objects (e.g., invoices with line items).
I’m happy to contribute a PR for this feature and would appreciate any feedback on scope, implementation details, design considerations, or if this is something you would actually want to be part of the engine at all.
Looking forward to your thoughts! 👋
The text was updated successfully, but these errors were encountered:
Allows tasks to process uploaded JSON files via Active Storage.
Sounds like a good idea to me.
Iterates over each object in a JSON array (or structured path).
Let's stick to iterating over JSON array elements, and skip the "structured path" idea (do you mean as in RFC 9535?).
Enables custom processing logic per item, similar to csv_collection.
What did you have in mind here? For CSV we're passing the options, but the custom processing logic is just an option of CSV. Similarly for JSON, we should just pass the options to JSON.parse.
The
maintenance_tasks
engine is an excellent tool for managing data migrations in Rails apps, especially with its built-in support for CSV via csv_collection. However, its current focus on flat, tabular data limits its usability in scenarios where JSON is the preferred format—particularly for data with nested or hierarchical structures.Problem:
CSV is flat: Difficult to represent nested attributes or complex relationships without workarounds.
Proposal:
json_collection
Introduce a
json_collection
that:csv_collection
.Use Cases
I’m happy to contribute a PR for this feature and would appreciate any feedback on scope, implementation details, design considerations, or if this is something you would actually want to be part of the engine at all.
Looking forward to your thoughts! 👋
The text was updated successfully, but these errors were encountered: