-
Notifications
You must be signed in to change notification settings - Fork 23
Description
Our GitHub continuous integration (CI) tests attempt to strike a balance between comprehensiveness (platforms, python versions, etc.) and speed to facilitate rapid iteration. Currently we have a test workflow that runs on every push to a PR and a test-comprehensive workflow that runs whenever a PR is merged into main. The difference between the two is the number of platforms and python versions that are tested.
Several recent PRs (#258 , #277 ) greatly expanded the scope of our unit tests for activity coefficients with different salts (and caught several bugs in the process), but these changes increased the time it takes to run typical CI tests on a PR (the test workflow) from ~1-2 minutes to 15-20 minutes.
This is not an ideal situation. While I like the comprehensiveness of the parameterized tests, I also appreciate the value of FAST CI tests when one is working on a PR.
We should review
- Our Github workflow strategy (e.g.,
testvstest-comprehensiveworkflows, no. of parallel tests, etc.), and - The parameterization of the salt / activity coefficient tests
To make sure that all combinations are necessary and to find a better balance between rapid testing within PRs and comprehensive testing when new changes are pushed to main.