Skip to content

Improve support for bulk deletes (when reprocessing lots of companies) #806

Open
@ianco

Description

@ianco

Currently there are two methods of deleting companies from OrgBook (in order to reprocess the data from BC Registries):

  1. Deleting individual companies using the "manage" command (executed via the orgbook-configurations project)
  2. Manually deleting the company information from the OrgBook database, where there are a significant number of companies to reprocess

There are some drawbacks to the second approach:

  • the OrgBook search indexes are not updated, so these must be re-run once the data updates are completed (this can take several days for a full re-index)
  • the credentials are not deleted from the OrgBook wallet

It would be preferable if the first approach supported bulk deletes. This would be a fairy straightforward update:

  • allow multiple company numbers to be input, comma-delimited (e.g. manage.py delete_topic BC123456,FM123456,A123456)
  • convert the input topic(s) to a list, and iterate through the list
  • delete each topic

It's not possible to "group" the companies within the script, since the wallet credentials must be deleted individually, and the Django framework is used to perform the actual delete.

The execution time may be too much if there are a lot of companies being deleted - the processing time has been added to the script in PR #805, so this can be measured/estimated in the future.

It is expected that there will be 3 methods required to delete OrgBook companies:

  1. Deleting individual companies using the "manage" command (executed via the orgbook-configurations project)
  2. Deleting some intermediate number of companies using the "manage" command (max number to be determined based on timings)
  3. Manually deleting the company information from the OrgBook database, where there are a significant number of companies to reprocess (more than the manage script can handle in a reasonable time)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions