Collection of mGAP/JBrowse/Search Issues

[bbimber]

Below is a list of open mGAP-related issues we should tackle:

• - Should update to the latest version of JBrowse and any other relevant dependencies
• - When opening a JBrowse feature popup, that dialog is currently getting cut off since it isnt handling overflow. You fixed this as a one-off for our Variant popup, but built-in popups (like when you click a gene) are still cut off. Try to figure out what causes this, and potentially put a PR into JBrowse itself, if that is a more robust solution.
• - We need to re-label many of the filters to make them more end-user intuitive. Notably, 'in-set' should be something like 'equals one of'. See a LabKey filter dialog (present for most grids) for ideas on terminology.
• - Related to the above, we should refactor the filter types to be more object-oriented and consolidate that code into something like an Enum or helper class. This should have a single method to translate from the string ("e.g., 'in-set') to a full Filter object, which could store a value for "Label", etc. All serialization should be handled in one place, with most code passing around Filter objects not strings. Related to: Coherent plan around serializing JBrowse filters/location to URL #218
• - This probably should include a class for the field itself (which is not aware of the column), and the logic of how it chooses the operator set and default operator could allow the columnInfo object to provide defaults and/or choose behaviors (like whether multi-value is allowed).
• - We need a consistent story and behavior about how and when we handle multi-valued fields, and how this interacts with the filter UI. Object-oriented consolidated code should help. Related to this issue: Free text search: multi-valued numeric fields #192. As a general rule, if it's reasonable to let people search on more than one value and treat it like an 'OR', we probably should let them. Again, the behavior of the LabKey grid filter dialog is something we should emulate, as much as reasonably possible.
• - Filters should chose a reasonable default value for their operator, including based on their existing value (i.e. if there is no value, it picks a reasonable default and if there is data, choose the right operator for that). The field should be able to provide a 'defaultOperator' for itself as well.
• - Need to make sure TSV export works robustly, including with pagination and very large datasets. For testing, I think it might be useful to set the page size very small (which forces the data into a multi-paged form) and make a panel of test-cases that toggle pages, do export, do filtering, etc. We can test this manually on mGAP as well. My sense from limited testing is that pagination is not always returning what it should after you leave the first page.
• - On both the original variant grid and newer search grid, the user can be overwhelmed by the shear number of columns. Can we provide some sort of type-ahead feature to choose columns to show?
• - Similar to above, can we provide a popup/hover type of thing that displays the column description? Maybe put a "?" after the text. If this is not possible, we could consider having a link/button elsewhere on the page that opens a model showing a scrolling table of the available fields and their description. It would be extra helpful if this popup had buttons next to each field like "add to grid".
• - Can you make the pagination section page total use commas? These are fairly big numbers.
• - For both grids, it would be useful if the grid allowed users to resize the columns with the mouse. At least see what MUI supports on this.
• - On mGAP, it takes something like 10 seconds to load the initial variant search page. A simplistic analysis shows it's ~1s to call getSession and getIndexedFields (which are in parallel), and about 10s for the initial lucene query. It might be worth some lucene profiling to see if there are simple ways to improve search. We could give this more threads, or look at how we generate the index. This is one guide I found from googling: https://cwiki.apache.org/confluence/display/lucene/ImproveSearchingSpeed.
• - Double check whether this issue is fixed: Variant Search sort on position #272
• - Review this issue: Provide easy 'Share" button on genome browser and variant table that copies a stable URL to the clipboard #292
• - This would be a very user-friendly feature: Free text search: handling chromosome/start #188
• - JBrowse supports the concept of highlighting a region in the current view (https://jbrowse.org/jb2/blog/2024/03/06/v2.10.3-release/). You give it the coordinates to specify what to highlight. We should support a URL param like 'highlight=1..X-Y", and pass that to JBrowse. See my comments on this thread: Create general "highlight"/"marker" indicators GMOD/jbrowse-components#599. I think this is potentially trivial, but we need to figure out the right entry point into the JBrowse LGV object.
• - As-is, VcfTabixAdapter.ts is a straight copy-paste from JBrowse. We should see if they will export this, so we dont need this hack. See suggestion here about getSubAdapter: Export VcfTabixAdapter? GMOD/jbrowse-components#4935 (comment)
• There are some larger tasks that could make big differences:
• - See here for allowing dual VCF data sources: Allow variant tracks to use one URL to show overview and another for the details popup discvr-components#38
• JBrowseLuceneSearch.java should stream the search result JSON, rather than generate the object in memory