Usable cross-identification interface

[Kraysent]

Problem

Currently the cross-identification interface is inaccessible to an administrator that does not have direct access to a DB. I want to create a usable interface.

For all the text below record_id and object_id are used interchangeably and mean the ID of the record from original layer 0 tables (which is distinct from PGC which is an ID of an aggregated layer 2 object).

Requirements

Functional:

• Ability to view statistical overview of the cross-identification - totals for new, existing and collided records after and during the completion of cross-identification task.
• Ability to list and filter results of crossmatch by:
  • status (new, existing, collided)
• Ability to view results of crossmatch for individual records with all available information. This includes:
  • Potential PGC candidates for the record
  • Marking information about the record
  • Original column info about the original record
  • Layer 2 information about all PGC candidates
• Ability to choose and submit one of PGC candidates as a result of the cross-identification
• Ability to override PGC candidates and choose either your own PGC number or forcefully mark object as new
• After the submission of the result of a particular record the page should automatically move to the next object without duplication

Non-functional

• Auth-gated ability to both viewing and submitting of record crossmatch results
• Ease of addition of new filters
• Viewing crossmatch result of a particular record loads in under 1 second (including loading of the web-interface)

Proposed solution

Backend

We already have a table with the results of crossmatch. It has a following schema:

• object_id - id of the original record
• status- crossmatch status
• metadata - custom data associated with the status. Nothing for new, single PGC for existing and possible PGC candidates for collided.

I want to extend this schema by adding two new fields:

• decision: enum[automatic, manual] - who decided a particular result of crossmatch. automatic by default meaning that automated crossmatch algorithm assigned this result. manual will be set when the user overrides the result or decides the collision of a record. This will be used for filtering
• created_by: int - FK to a user ID from users table. Is set when a user runs crossmatch script or overrides the decision from the interface.

To accomplish functional requirements we need to create new handlers and alter some existing ones:

• Statistical view: we already have GET /v1/table method that returns information about the layer 0 table. If available, we can return information with the statistical overview of the crossmatch results.
• Listing and filtering of the crossmatch results: new handler - GET /v1/records/crossmatch that will return list of crossmatch results. Inputs are: table_name, status (collided by default), decision (optional, all by default). In future more filters might be added. The output will contain list of records, each one should contain record id (aka object_id), status, metadata about the status and marked information such as name, coordinates and other available catalogs. It should also have units of measurement similar to how the GET /v1/object of data API does (most likely will need to reuse the same schema). Method is paginated meaning it has page and page_size parameters as well.
• Viewing individual crossmatch results: GET /v1/records/{object_id}/crossmatch method that should take record id (object_id) as input and return verbose information about the record and its crossmatch results. This information should include:
  • record's marked information, including all marked catalogs with their units
  • record's original data, including its units
  • status of crossmatch
  • for existing and collided records - list of candidate objects each of which contain the PGC number and layer 2 information about the object (coordinates, names, etc.)
• Result submission: POST /v1/records/{object_id}/crossmatch that accepts record id and optional PGC number. If PGC is specified, object is marked as existing and assigned a PGC. If no PGC is specified, object is marked as new. This method does not check whether PGC provided was the candidate or not.

Frontend

• Statistical view: /table/{table_name} page will have one more sub-table showing results of a crossmatch and a button Go to results that redirects to listing of crossmatch results.
• Listing and filtering of the crossmatch results: /crossmatch page with query parameters: table_name, status (optional), decision (optional) and pagination. Initially it would have several dropdowns for status and decision. Target solution will require some fields to input table_name (so user does not have to manually enter it in address bar in query parameters) and filter by name and coordinates but the exact view of those is out of scope of this ticket.
• Viewing individual crossmatch results: /records/{record_id}/crossmatch - this page has Aladin embed centered on the coordinates of the original record and separate marks for candidate PGC numbers. It also has a table for comparison of marked parameters for the record itself and all candidate PGCs - columns are parameters and rows are candidates. Each candidate has a button that allows user to select it as a submission for later.
• Result submission: when user selects an object, Submit button activates and, when user clicks on it, it sends POST request to the server.
• Going to the next record/skip record: the crossmatch result page also contains Skip button that simply switches user to the next object. Skip button is only active if special query parameters are present on the page. These parameters mirror query parameters of the /crossmatch page (without the pagination) and allow Skip and Submit buttons to determine what is the next object to switch to. Apart from /crossmatch query parameters it also contains offset parameter that determines which object to take and is basically a page parameter for the /crossmatch pagination with page_size=1 and decision=automatic.

Sub-issues:

• Create POST /v1/records/{object_id}/crossmatch endpoint for crossmatch result submission
• Create /crossmatch page for listing and filtering results and add button to /table/{table_name} page
• Create /records/{record_id}/crossmatch page for viewing the results of crossmatch
• Add submissions and skips to /records/{record_id}/crossmatch page