Flarelist anc #431
Merged
Flarelist anc #431
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nicHoch
force-pushed
the
flarelist_ANC
branch
from
80cc34b to
9952477
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #431 +/- ##
==========================================
- Coverage 72.23% 69.25% -2.98%
==========================================
Files 78 76 -2
Lines 8085 8100 +15
==========================================
- Hits 5840 5610 -230
- Misses 2245 2490 +245 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
nicHoch
force-pushed
the
flarelist_ANC
branch
2 times, most recently
from
4fe20e4 to
88d949c
Compare
Add support for trigger scaling in compression mode S=0,k=0,M=7 (i4Ds#377) * Unscale data when compressed with mode S=0,k=0,M=7 * QL - LC, BKG, SPECTRA * BSD - CPD, SCPD, VIS and SPEC bump all product versions to 2 (i4Ds#378) * bump all product versions to 2 * roleout IDB version 2.26.27 * fix test with new version number in mock object * fix test * fix for latest IDB as fallback Fix duplicated hk time bin bug Ddpdupdates2 (i4Ds#380) * bump all product versions to 2 * prep for L2 ddpd * integrate IDB release 2.26.38 * update for common CONF submodule * fir text for latest IDB to 2.26.38 Config fix (i4Ds#381) * bump all product versions to 2 * fix module path Fix bugs found when processing new scaled trigger data. (i4Ds#382) * Fix bug processing new scaled trigger data. * Allow for QL file to unscale by packet * Fix bug that create extra (replica) triggers in Spectrogram Publish_L2 / RID LUT Manager (i4Ds#385) * bump all product versions to 2 * RID LUT data manager update and use in processing for trigger descaling * Update stixcore/io/RidLutManager.py * fixed API mock via static files on pup99 --------- Co-authored-by: Shane Maloney <maloneys@tcd.ie> Add warning if trigger scaling changes in request (i4Ds#388) End2end rid lut manager (i4Ds#390) * bump all product versions to 2 * add copy and parent option * fix missing init of singleton bump all product versions to 2 (i4Ds#389) Bump conf version (i4Ds#391) * bump all product versions to 2 * bump common version Flexible way to fill DATAMAX/MIN BUNIT EXPOSURE in FITS header keywords (i4Ds#393) * bump all product versions to 2 * add properties for dedicated fits header keywords with class inhe. / override * add test and minor fixes after review * fix XPOSURE/XPOMAX header error * fix tests for dmax... * relax end2end fitsdiff * less tolerance in end2end test Solo anc stix ephemeris (i4Ds#399) * bump all product versions to 2 * add more FITS header keywords * fix for ANC also get fits updated headers while publishing Upgrade_watchdog (i4Ds#401) * bump all product versions to 2 * bump version * fix version label * pin common CONF version for end2end test * ignore VERS_CFG in end2end testing * add test for TM folder observing * bump watchdog==6.0.0 * integrate review fixes Update docks (i4Ds#409) * bump all product versions to 2 * update documentation * add doc for end to end tetsing * fix format * add doc for manual (re)processing of fits products E2e fix sym add (i4Ds#412) * bump all product versions to 2 * dep np.bool8 * disable test Get rid reason/scaling lookup error handling (i4Ds#411) * bump all product versions to 2 * add global error detection in pipeline logger (not just main process) * add exception handling for get_reason daily processing pipeline (i4Ds#405) * bump all product versions to 2 * initial * daily processing task * add singleStep * move ANC-aspect from pipeline to daily_pipeline * removed requirement for the moment * fix i4Ds#283 and option to process only latest fits file versions * update ddpd scripts * fix circular imports * test git dep * skio test on win * skip test on win/mac * Update stixcore/io/FlareListManager.py Co-authored-by: Shane Maloney <maloneys@tcd.ie> * fix format error --------- Co-authored-by: Shane Maloney <maloneys@tcd.ie> Add Zenodo file (i4Ds#413) * Add Zenodo author information Update high resolution transmission to match IDL defaults (i4Ds#414) Pin major and minor versions of some packages (i4Ds#415) V1.5.1_fixes (i4Ds#420) * bump all product versions to 2 * fix i4Ds#417 * fix i4Ds#418 * daily report log files could have different names if the scripts run over multiple days * fix i4Ds#419 * end2end test: better report if no corresponding file found * rewrite pipeline to cronjob based approach i4Ds#422 * cleanup before merge Fix bug in spectrogram time bin calculations (i4Ds#423) * Only use the closing time offset (NIX00269) for the last science substrure Update .zenodo.json Maintenance update project and dependencies (i4Ds#428) * Update project use project.toml and move config into own files * Fix most of the tests * ruff-check, ruff-format and codespell * Some spelling mistakes left need to check the meaning * More updates and fixes * Few more small fixes * Fix doc build * Revert unintentional spelling correction in IDB sql script. * Update tox test dep name * Add version.py to git * Skip unclear test add skipped test for later * Update workflow python version * Add remote data to github CI bump all product versions to 2 Add support for trigger scaling in compression mode S=0,k=0,M=7 (i4Ds#377) * Unscale data when compressed with mode S=0,k=0,M=7 * QL - LC, BKG, SPECTRA * BSD - CPD, SCPD, VIS and SPEC bump all product versions to 2 (i4Ds#378) * bump all product versions to 2 * roleout IDB version 2.26.27 * fix test with new version number in mock object * fix test * fix for latest IDB as fallback Fix duplicated hk time bin bug Ddpdupdates2 (i4Ds#380) * bump all product versions to 2 * prep for L2 ddpd * integrate IDB release 2.26.38 * update for common CONF submodule * fir text for latest IDB to 2.26.38 Config fix (i4Ds#381) * bump all product versions to 2 * fix module path Fix bugs found when processing new scaled trigger data. (i4Ds#382) * Fix bug processing new scaled trigger data. * Allow for QL file to unscale by packet * Fix bug that create extra (replica) triggers in Spectrogram Publish_L2 / RID LUT Manager (i4Ds#385) * bump all product versions to 2 * RID LUT data manager update and use in processing for trigger descaling * Update stixcore/io/RidLutManager.py * fixed API mock via static files on pup99 --------- Co-authored-by: Shane Maloney <maloneys@tcd.ie> Add warning if trigger scaling changes in request (i4Ds#388) End2end rid lut manager (i4Ds#390) * bump all product versions to 2 * add copy and parent option * fix missing init of singleton bump all product versions to 2 (i4Ds#389) Bump conf version (i4Ds#391) * bump all product versions to 2 * bump common version Flexible way to fill DATAMAX/MIN BUNIT EXPOSURE in FITS header keywords (i4Ds#393) * bump all product versions to 2 * add properties for dedicated fits header keywords with class inhe. / override * add test and minor fixes after review * fix XPOSURE/XPOMAX header error * fix tests for dmax... * relax end2end fitsdiff * less tolerance in end2end test Solo anc stix ephemeris (i4Ds#399) * bump all product versions to 2 * add more FITS header keywords * fix for ANC also get fits updated headers while publishing Upgrade_watchdog (i4Ds#401) * bump all product versions to 2 * bump version * fix version label * pin common CONF version for end2end test * ignore VERS_CFG in end2end testing * add test for TM folder observing * bump watchdog==6.0.0 * integrate review fixes Update docks (i4Ds#409) * bump all product versions to 2 * update documentation * add doc for end to end tetsing * fix format * add doc for manual (re)processing of fits products E2e fix sym add (i4Ds#412) * bump all product versions to 2 * dep np.bool8 * disable test Get rid reason/scaling lookup error handling (i4Ds#411) * bump all product versions to 2 * add global error detection in pipeline logger (not just main process) * add exception handling for get_reason daily processing pipeline (i4Ds#405) * bump all product versions to 2 * initial * daily processing task * add singleStep * move ANC-aspect from pipeline to daily_pipeline * removed requirement for the moment * fix i4Ds#283 and option to process only latest fits file versions * update ddpd scripts * fix circular imports * test git dep * skio test on win * skip test on win/mac * Update stixcore/io/FlareListManager.py Co-authored-by: Shane Maloney <maloneys@tcd.ie> * fix format error --------- Co-authored-by: Shane Maloney <maloneys@tcd.ie> Add Zenodo file (i4Ds#413) * Add Zenodo author information Update high resolution transmission to match IDL defaults (i4Ds#414) Pin major and minor versions of some packages (i4Ds#415) fix i4Ds#417 fix i4Ds#418 daily report log files could have different names if the scripts run over multiple days fix i4Ds#419 end2end test: better report if no corresponding file found rewrite pipeline to cronjob based approach i4Ds#422
nicHoch
force-pushed
the
flarelist_ANC
branch
from
88d949c to
9d13144
Compare
fix tests fix tests fix tests
nicHoch
force-pushed
the
flarelist_ANC
branch
from
c925de7 to
34f27b8
Compare
samaloney
reviewed
Oct 2, 2025
samaloney
reviewed
Oct 2, 2025
samaloney
reviewed
Oct 2, 2025
samaloney
reviewed
Oct 2, 2025
samaloney
reviewed
Oct 2, 2025
samaloney
reviewed
Oct 2, 2025
Collaborator
samaloney
left a comment
samaloney
left a comment
There was a problem hiding this comment.
Lots going on here, some of it looks still in progress in particular the flare list stuff, so hard to fully understand but all seems reasonable
Collaborator
Author
|
@samaloney please do a quick review again and approve if possible so that that can be merged finally |
samaloney
approved these changes
Oct 16, 2025
Collaborator
samaloney
left a comment
samaloney
left a comment
There was a problem hiding this comment.
Hard to check everything as it a large PR but looks reasonable.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
handels flare lists
but also LL3 data