[cimi] support bulk delete of resources from a collection

[konstan]

Background: GNSS will be using Nuvla as its data base for storing and deleting ~10K * N SO documents per hour. (N is the number of stations - in the projects for 8 month there will be 3 stations). It would be good to have bulk delete operation based on query.

Propose where to put it in the API and make sure it doesn’t conflict with any of the current CIMI patterns.

Options:

1. DELETE a set of objects in a collection based on CIMI filter (eg. $filter=resource:timestamp>='time_start' and resource:timestamp<='time_end')
2. “bulk delete” operation on collections (providing list of objects to delete).

[loomis]

This can be implemented as a DELETE request to the corresponding collection resource without disrupting the standard CIMI patterns. (Typically a DELETE on the collection is not permitted.) The Elasticsearch implementation of this would be straightforward, as the Elasticsearch API supports an asynchronous "delete by query" feature. As this is asynchronous, this could be represented by a job that could keep track of the status of the delete action.

However to ensure that the constraints of the ACL are respected while using the database-level "delete by query", the "delete" access rights (at least) need to be stored with the document. Currently, only the "view" access right, which facilitate the CIMI search/query features, is stored. Supporting this feature will require a significant re-implementation of the ACL/database bindings (probably also with a new format for the ACL). After the re-implementation of this, the entire database will have to be re-indexed.

Although I'm generally in favor of doing this, it will require significant work that will disrupt other development. There needs to be a consensus before starting to implement these changes.

[loomis]

As a prerequisite for this, see the proposal for changes to the ACLs.