feat: Add T2 attribution module for diagnosing temperature changes #918
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
🤖 I've automatically formatted the code in this PR using:
Please pull the latest changes before making further edits. |
1 similar comment
|
🤖 I've automatically formatted the code in this PR using:
Please pull the latest changes before making further edits. |
sunt05
force-pushed
the
feat/t2-attribution-module
branch
from
643db7d to
e42529e
Compare
|
🤖 I've automatically formatted the code in this PR using:
Please pull the latest changes before making further edits. |
sunt05
force-pushed
the
feat/t2-attribution-module
branch
2 times, most recently
from
525ea60 to
833d7dd
Compare
|
🤖 I've automatically formatted the code in this PR using:
Please pull the latest changes before making further edits. |
sunt05
force-pushed
the
feat/t2-attribution-module
branch
from
a85dddc to
64d099b
Compare
Preview Deployed
Note This preview is ephemeral. It will be lost when:
To restore, push any commit to this PR. |
Add a new diagnostic module that decomposes near-surface temperature differences between scenarios into physically attributable components using Shapley value decomposition. Features: - attribute_t2(): Compare two scenarios, decompose delta-T2 into flux, resistance, and air property contributions - diagnose_t2(): Automatic anomaly detection with methods: 'anomaly', 'extreme', 'diurnal' - AttributionResult class with plot() method (bar, diurnal, line, heatmap) - Hierarchical flux breakdown into Q*, QE, dQS, QF components - Exact closure guaranteed by Shapley decomposition Use cases: - Debug unexpected T2 values in simulations - Understand why green infrastructure cools cities - Quantify physical mechanisms driving temperature changes 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Co-authored-by: sunt05 <1802656+sunt05@users.noreply.github.com>
Add multi-variable attribution capability following the same Shapley decomposition framework used for T2. The q2 (2m specific humidity) attribution uses latent heat flux (QE) and the scale factor gamma = 1/(rho*Lv) to decompose humidity differences into physical mechanisms. New functions: - attribute_q2(): Decompose q2 differences between two scenarios - diagnose_q2(): Automatic anomaly detection for humidity - attribute(): Generic dispatcher for T2/q2 - diagnose(): Generic dispatcher for T2/q2 - _cal_gamma_humidity(): Scale factor for latent heat - _cal_r_eff_humidity(): Back-calculate moisture resistance The design maximises code reuse - both T2 and q2 follow the same flux-gradient physics pattern (X = X_ref + r * F * gamma). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Co-authored-by: sunt05 <1802656+sunt05@users.noreply.github.com>
Include the new attribution module in the meson build system. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Add comprehensive test suite for T2/q2 attribution functions - Improve error handling with numpy errstate for division operations - Add input validation for required DataFrame columns - Add warnings when forcing data not provided (affects accuracy) - Add warnings when significant data loss during index alignment - Add logging for low-flux timesteps producing NaN values - Improve docstrings with mathematical references (Owen 1972) - Fix Celsius to Kelvin conversion (273.16 -> 273.15) - Fix British English spelling in tutorial documentation
Restructure _attribution.py into _attribution/ package with separate modules for better maintainability. Add wind speed (U10) attribution alongside existing T2 and Q2 support. Co-Authored-By: Claude <noreply@anthropic.com>
Co-authored-by: sunt05 <1802656+sunt05@users.noreply.github.com>
Update test imports to reflect the refactored attribution module which is now organised as a package with separate submodules for core Shapley functions, physics calculations, and helper utilities. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Change df_forcing_A/df_forcing_B from Optional to required parameters - Remove fallback approximation code that used T2/q2 as proxy values - Update docstrings to clarify forcing requirements (Tair, RH, pres) - Ensure numpy array conversion for cal_gamma_heat results - Update tests with df_forcing fixtures for all attribution calls - Remove test_attribute_t2_warns_on_missing_forcing (no longer applicable) T2/q2 attribution requires reference temperature/humidity from forcing data for accurate Shapley decomposition. U10 does not require forcing as all profile components derive from SUEWS output variables. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
The AttributionResult.plot() method now validates that the index is a DatetimeIndex before attempting to access .hour or .month attributes. This prevents AttributeError when calling plot(kind='diurnal') or plot(kind='heatmap') on aggregate results from diagnose_* functions. Also updates the tutorial notebook to use kind='bar' for the aggregate diurnal comparison result, which is the correct plot type for single-row attribution results. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
#1029) Convert attribution-tutorial.ipynb to text-based attribution_tutorial.py using percent-format cells. The .ipynb is now auto-generated during docs build via jupytext, improving maintainability: - Text-based source enables clean git diffs and trivial merges - Makefile auto-generates .ipynb from .py during `make docs` - Added jupytext to dev dependencies - .ipynb excluded from git (build artifact only) This pilots the approach proposed in #1029 for migrating all tutorials.
sunt05
force-pushed
the
feat/t2-attribution-module
branch
from
e73e740 to
c4454c6
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Add a new diagnostic module that decomposes near-surface temperature (T2) differences between scenarios into physically attributable components using Shapley value decomposition.
Key Features
attribute_t2(): Compare two SUEWS scenarios, decompose ΔT2 into:diagnose_t2(): Automatic anomaly detection for single-run debugging:'anomaly': Detect timesteps > N σ from daily mean'extreme': Compare top/bottom 5% vs. middle 50%'diurnal': Compare afternoon peak vs. morning baselineAttributionResultclass with:__repr__showing percentagesplot()method with 4 visualisation modes: bar, diurnal, line, heatmapto_dataframe()for further analysisMathematical Foundation
Uses exact Shapley decomposition for the triple product T2 = r × QH × γ:
Use Cases
Files Changed
src/supy/util/_attribution.pysrc/supy/util/__init__.pydocs/source/tutorials/python/attribution-tutorial.ipynbdocs/source/tutorials/python/tutorial.rstTest plan
__repr__generates formatted output🤖 Generated with Claude Code