Skip to content

Feature Request: Add json_collection for Native JSON Data Handling #1205

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sabljak opened this issue Apr 18, 2025 · 1 comment
Open

Feature Request: Add json_collection for Native JSON Data Handling #1205

sabljak opened this issue Apr 18, 2025 · 1 comment

Comments

@sabljak
Copy link

sabljak commented Apr 18, 2025

The maintenance_tasks engine is an excellent tool for managing data migrations in Rails apps, especially with its built-in support for CSV via csv_collection. However, its current focus on flat, tabular data limits its usability in scenarios where JSON is the preferred format—particularly for data with nested or hierarchical structures.

Problem:

CSV is flat: Difficult to represent nested attributes or complex relationships without workarounds.


Proposal: json_collection

Introduce a json_collection that:

  • Allows tasks to process uploaded JSON files via Active Storage.
  • Iterates over each object in a JSON array (or structured path).
  • Enables custom processing logic per item, similar to csv_collection.

Use Cases

  • Migrating data from JSON columns in the database.
  • Handling product catalogs exported as JSON.
  • Processing arrays of deeply nested objects (e.g., invoices with line items).

I’m happy to contribute a PR for this feature and would appreciate any feedback on scope, implementation details, design considerations, or if this is something you would actually want to be part of the engine at all.

Looking forward to your thoughts! 👋

@etiennebarrie
Copy link
Member

  • Allows tasks to process uploaded JSON files via Active Storage.

Sounds like a good idea to me.

  • Iterates over each object in a JSON array (or structured path).

Let's stick to iterating over JSON array elements, and skip the "structured path" idea (do you mean as in RFC 9535?).

  • Enables custom processing logic per item, similar to csv_collection.

What did you have in mind here? For CSV we're passing the options, but the custom processing logic is just an option of CSV. Similarly for JSON, we should just pass the options to JSON.parse.

Feel free to open a PR!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants