-
Notifications
You must be signed in to change notification settings - Fork 91
Add unified CEBRA encoder: pytorch implementation #251
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add unified CEBRA encoder: pytorch implementation #251
Conversation
…ional models in _transform
add headers to new files
- header
- header
MMathisLab
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks @CeliaBenquet ! I went through and left comments for disucssion
stes
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good overall; left some comments!
- Implementation of the Mixin class for the masking: If I understood correctly, the only change is that this
apply_maskfunction is applied after loading a batch. This seems to be a change that could be minimally invasively applied not in the dataset, but actually in the data loader. Is there a good case why the datasets themselves need to be modified? - Discussion on where to place the decoders: currently in
cebra.models.decoders; are the decoders useful as "standalone" models? where are they currently used? based on that we could determine if we move them e.g. as standalone tointegrations - see other comments; mostly on class design, removing duplicated code, etc.
MMathisLab
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm! Just the one comment on kwargs seems critical to decide
stes
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remaining issues are:
- more tests (e.g. for changes to model, adaptations of sklearn API, ...)
- improved class design for masking
- improved class design for solver hierarchy
However, this can be done in follow-up contributions, this can be merged now. Attempt to clean up the commit message for sqashing:
* start tests
* remove print statements
* first passing test
* move functionality to base file in solver and separate in functions
* add test_select_model for multisession
* remove float16
* Improve modularity remove duplicate code and todos
* Add tests to solver
* Fix save/load
* Fix extra docs errors
* Add review updates
* apply ruff auto-fixes
* fix linting errors
* Run isort, ruff, yapf
* Fix gaussian mixture dataset import
* Fix all tests but xcebra tests
* Fix pytorch API usage example
* Make xCEBRA compatible with the batched inference & padding in solver
* Add some tests on transform() with xCEBRA
* Add some docstrings and typings and clean unnecessary changes
* Implement review comments
* Fix sklearn test
* Initial pass at integrating unifiedCEBRA
* Add name in NOTE
* Implement reviews on tests and typing
* Fix import errors
* Add select_model to aux solvers
* Fix tests
* Add mask tests
* Fix docs error
* Remove masking init()
* Remove shuffled neurons in unified dataset
* Remove extra datasets
* Add tests on the private functions in base solver
* Update tests and duplicate code based on review
* Fix quantized_embedding_norm undefined when `normalize=False` (#249)
* Fix tests
* Adapt unified code to get_model method
* Update mask.py
add headers to new files
* Update masking.py
- header
* Update test_data_masking.py
- header
* Implement review comments and fix typos
* Fix docs errors
* Remove np.int typing error
* Fix docstring warning
* Fix indentation docstrings
* Implement review comments
* Fix circular import and abstract method
* Add maskedmixin to __all__
* Implement extra review comments
* Change masking kwargs as tuple and not dict in sklearn impl
* Add integrations/decoders.py
* Fix typo
* minor simplification in solver
---------
Note, some comments in this PR overlap with
https://github.com/AdaptiveMotorControlLab/CEBRA/pull/168
and
https://github.com/AdaptiveMotorControlLab/CEBRA/pull/225
which were developed in parallel.
This PR adds a PyTorch implementation of a unified CEBRA encoder, which is composed of:
🚧 A preprint is pending "Unified CEBRA Encoders for Integrating Neural Recordings via Behavioral Alignment" by Célia Benquet, Hossein Mirzaei, Steffen Schneider, Mackenzie W. Mathis.
💻 A DEMO Notebook is available at: https://cebra.ai/docs/demos.html