Merge upstream 26 01 05 #149
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* refactor: another PyPSA v1 deprecation * `0.35.2` compatibility
Co-authored-by: lkstrp <lkstrp@pm.me>
* feat: add river-water and sea-water heat pumps * remove obsolete modules * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * feat: add copernicusmarine to envs * feat: Add option to disable progress bar in seawater data retrieval script * fix: Ensure temperature difference is non-negative in heat flow calculation * fix: Impose cut-off temperature for sea water heat * feat: Add temporal aggregation for river and sea water heat potential calculations as outputs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * chore: Add SPDX license identifiers to surface water heat potential scripts * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: Use no-verbose mode for wget when retrieving hera data * fix: typos * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: update chunking parameters for river and ambient temperature datasets to surpress warnings * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * doc:: update data bundle docs to include Copernicus seawater temperature dataset with source and licensing information * refactor: update seawater temperature data path to bundle, add file to bundle retrieval and remove unused retrieval script * feat: update seawater data retrieval * fix: import shapely correctly in sea_water_heat_approximator.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * refactor: streamline temporal aggregate merging * feat: add heat source mapping functionality and ambient air temperature processing * fix: correct region geometry buffering in SeaWaterHeatApproximator * feat: enhance heat source map with aggregation options for temperature and energy * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: remove seawater temperature data file from databundle retrieval * fix: update seawater temperature data source and improve region geometry handling * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * feat: add rule and script for plotting coefficient of performance (COP) profiles of heat pumps * fix: correct import paths for HeatSector and HeatSystemType in heat_system.py * fix: update output paths for COP profiles to use 'graphs' directory * feat: enhance COP data preparation to support multiple heat system types and improve plotting functionality * refactor: streamline COP data preparation and enhance interactive plotting functionality * feat: add region_dim parameter to prepare_cop_data function for enhanced flexibility * fix: improve validation for regions_onshore and enhance aggregate_type handling in plot_heat_source_map function * feat: add region_dim parameter to create_interactive_cop_plot function for improved region filtering * feat: add rules for bus carrier dispatch and annual energy balance plots in Snakemake workflow * refactor: update output paths for bus carrier dispatch plots and adjust memory resources in postprocess rules * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * use absolute imports (#1643) * use absolute imports * add release note * fix typo * Non-sequestered HVC Waste Allocation According to Population (#1625) * non-sequestered HVC waste allocation according to population * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * adding release note and correcting link efficiencies * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * moving locations of non-sequestered hvc to 'spatial' variable * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de> Co-authored-by: Micha <michaellindner@posteo.de> * fix: update fallback value for unusable water temperature from absolute zero to -1e9 * feat: enhance water heat potential models and improve data handling - Added district heating area buffer configuration in config.default.yaml - Updated plotting.default.yaml to include river and sea water heat sources - Enhanced build_sector.smk to incorporate district heating area data - Introduced retrieve_dh_areas rule in retrieve.smk for fetching district heating areas - Refactored river_water_heat_approximator.py and sea_water_heat_approximator.py for improved data validation and processing - Modified build_river_water_heat_potential.py and build_sea_water_heat_potential.py to handle district heating areas - Improved plot_heat_source_map.py for better data filtering and error handling * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: correct unit in heat power calculation * feat: clip COP bewlow 1 to 0 * refactor: simplify cop profile plotting * refactor: streamline heat source retrieval and improve plot info * refactor: simplify plot_bus_carrier_dispatch * refactor: enhance plot_bus_carrier_dispatch with bus name pattern and output directory * refactor: rename plot_bus_carrier_dispatch to plot_interactive_bus_balance and update output paths * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * refactor: reorder import statements for clarity * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * refactor: clean up whitespace in create_interactive_cop_plot function * feat: remove sea-water by default but add heat pump sources sea-water and geothermal to myopic, overnight, and perfect test configuration files * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * feat: update limited heat sources in sector.csv to include river_water and clarify data source requirements * refactor: remove outdated reference to heat_demand in data-retrieval.rst * doc: update release notes * doc: add interactive bus plotting to docs and test config * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: simplify output directory structure for heatmap timeseries in rule all * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update scripts/build_cop_profiles/BaseCopApproximator.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update scripts/build_ambient_air_temperature_yearly_average.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update rules/build_sector.smk Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update rules/build_sector.smk Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix: correct formatting of limited_heat_sources description in sector.csv * refactor: remove redundant comments and clean up coefficient of performance calculation * fix: update input_heat_source_temperature function to include 'ptes' in replace_names and adjust resource references * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: update input_heat_source_temperature and related rules for ptes naming conventions and output formatting * fix: add plotly dependency to environment.yaml and linux-pinned.yaml * fix: update bus_name_pattern in plotting configuration to None * fix: update bus_name_pattern in plotting configuration to None * compress hera data after download * use curl instead of wget and nccopy * refactor: revert back to uncompressed wget * bugfix: add existing renewables consistent with renewable_carriers list (#1717) * feat: include existing capacities only for included renewable carriers * bugfix: only add solar rooftop in distribution grid when solar included in config * revert mock snakemake changes * refactor: change to if not continue style to reduce diff * Add offwind float to CCL (#1725) * Fix: Invalid aquifer shapes in build_ates_potentials (#1724) * fix: ensure aquifer geometries are valid in build_ates_potentials.py * fix: add release note for fixing invalid aquifer shape geometries in build_ates_potentials.py * feat: use bundle hera data for test cutout * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: pass heat_pump_cop_approximation configs correctly (#1729) * fix: forward heat_pump_cop_approximation config to CentralHeatingCopApproximator * refactor: set correct heat source for ptes boosting and include titel/link to source for min_delta_t_lift --------- Co-authored-by: Amos Schledorn <60692940+amos-schledorn@users.noreply.github.com> * Fix ``plot_hydrogen_network`` : storage + Added CCL explantions in the configtables (#1727) * Add offwind float to CCL * Update plot_hydrogen_network.py Fix the code to plot the storage * Update release_notes.rst * Update solving.csv Add precision for the CCL constraint (not working with a time resolution higher than 1h) + add indication on how to use agg_p_nom_limits * fix: update method call to use 'cop()' instead of 'approximate_cop()' in get_cop function * fix: remove unnecessary parentheses in cop() method call in get_cop function * fix: update retrieve_hera_data rule to use config_provider for snapshot year in output paths * fix: update retrieve_hera_data rule to use snapshot_year directly in output paths * feat: introduce dh_area building and improve handling of missing dh_area countries * update config * doc: improve comments * update gitignore * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * feat:improve performance * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix:p_max_pu and p_nom_max for geothermal heat source * fix: update heat source handling in add_heat function * fix: remove unused heat pump sources from configuration * fix: add shipping sector and update zero cost settings for heat sources * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * doc:update release notes and docs * fix: enable CO2 sequestration potential check in add_co2_tracking function * refactor: type hints and improved documentation * refactor: enhance type hints and improve documentation in SurfaceWaterHeatApproximator * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update config * feat: reintroduce default target for the 'all' rule * refactor: update default parameters in RiverWaterHeatApproximator * fix: correct snapshot retrieval method in retrieve.smk * fix: improve handling of zero-cost heat sources in add_heat function * fix: update mock_snakemake call to use correct script name for sea heat potential * fix: correct default_target case in all rule and refactor snapshot_year assignment * fix: ensure temporal aggregate is only deleted when generated in get_regional_result function * fix: fix static config handling * fix: update default min_outlet_temperature in RiverWaterHeatApproximator to 1C * fix: update input parameters for build_river_heat_potential rule and correct mock_snakemake call * feat: wrap heat_source_map output into input function * fix: remove comment from generate_temporal_aggregates in plotting configuration * Revert "feat: wrap heat_source_map output into input function" This reverts commit e6306caf47f17ef087eed1d6ae73a78f39ac3fc1. * fix: update heat source map configuration references in Snakefile and plotting defaults * Update doc/configtables/sector.csv Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de> * fix:only plot heat source maps for activated heat pump sources * fix: correct description for enabling temporal aggregate maps for heat sources * fix:activate interactive_bus_balance ("*central heat*") in test configs * fix: improve warning message for missing heat source cost data * fix: replace print statements with logger for missing countries handling * fix: add bokeh and folium to environment dependencies * fix: update heat source map configuration for river, sea, and ambient air sources * fix: heat pump sources in test configs * fix: update heat source map configuration to use 'air' instead of 'ambient_air' * fix: update docstring formatting for RiverWaterHeatApproximator class * fix: update URLs for river discharge and ambient temperature data to V2 * fix: simplify power calculation constants in SurfaceWaterHeatApproximator * fix:cosmetic updates * Update doc/configtables/plotting.csv Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de> * Update doc/configtables/sector.csv Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de> * fix: update interactive bus balance configuration in plotting.csv * fix: update release notes * fix: remove unused dynamic inputs for hera data retrieval * fix: rename generate_temporal_aggregates to enable_heat_source_maps * Update rules/retrieve.smk Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de> * fix: update thread configuration for build_river_heat_potential rule * fix: update HERA data retrieval URLs to version 1.0 * fix: improve readability of input_heat_source_temperature function * fix:use move/storage + year wildcard in hera data retrieval * fix: update pypsa version to 0.33.0 in environment.yaml * fix: pypsa options usage * fix: pin `pypsa>=0.34.0` * Update locked environment files for all platforms (#1832) Co-authored-by: lkstrp <62255395+lkstrp@users.noreply.github.com> * feat: support multi-year snapshots * fix:re-add ptes temperature to build_cop_profiles rule * fix: update output filenames for ptes direct utilisation and temperature profiles * fix: update output filename for ptes direct utilisation profiles to include 's_' prefix * fix: correct indentation for hydrogen_turbine in sector configuration * feat: automate seawater temperature retrieval for custom weather years * refactor:simplify weather data handling through xr.open_mfdataset * feat: add rule to retrieve seawater temperature for 2013 and improve error handling in retrieval script * fix: update output file naming conventions in ptes operations * feat: enhance documentation and type hints in data retrieval and plotting scripts * fix: update output filename for ptes direct utilisation profiles to include 'base' * feat: conditionally define seawater temperature retrieval rules based on default cutout configuration * fix: rename seawater temperature retrieval rule for 2013 * fix: update seawater temperature output filename to reflect 2013 data * fix: update seawater temperature output filename for test cutout * fix: update seawater temperature data for test cutout * fix: update seawater temperature file references to 2013 data * fix: update seawater temperature references for test cutout * fix: streamline seawater temperature retrieval for test cutout and enhance logging * fix: remove benchmark logging for seawater temperature retrieval * feat: rename build_cop_approximator submodules to snakecase * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Resolve merge conflicts * refactor: update district heating area configuration and improve heat pump cost handling * fix: update rule name in mock_snakemake for testing in build_dh_areas script * refactor: remove unnecessary comment to avoid circular imports in input_hera_data function * refactor: update docstring for RiverWaterHeatApproximator to clarify parameters * refactor: clarify parameters in docstring for RiverWaterHeatApproximator --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Lukas Trippe <lkstrp@pm.me> Co-authored-by: Toni <153275395+toniseibold@users.noreply.github.com> Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de> Co-authored-by: Micha <michaellindner@posteo.de> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Daniel Rüdt <117752024+daniel-rdt@users.noreply.github.com> Co-authored-by: JuliaSvn <144602424+JuliaSvn@users.noreply.github.com> Co-authored-by: TomKae00 <144129436+TomKae00@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: lkstrp <62255395+lkstrp@users.noreply.github.com>
* Update locked environment files for all platforms * trigger CI --------- Co-authored-by: amos-schledorn <60692940+amos-schledorn@users.noreply.github.com> Co-authored-by: Amos Schledorn <a.schledorn@tu-berlin.de>
* Add option to set minimum unit dispatch for electrolysis * electrolysis p_min_pu documentation
Bumps the github-actions group with 2 updates: [actions/upload-artifact](https://github.com/actions/upload-artifact) and [actions/download-artifact](https://github.com/actions/download-artifact). Updates `actions/upload-artifact` from 4 to 5 - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](actions/upload-artifact@v4...v5) Updates `actions/download-artifact` from 5 to 6 - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@v5...v6) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-version: '5' dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions - dependency-name: actions/download-artifact dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Modified building network script to allow for islanded network building (e.g. just Norway). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Lukas Trippe <lkstrp@pm.me> Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
* Fix by specifying axis * Add release notes --------- Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
* feat: introduce build_cost_data * refactor: bring load_costs features into build_cost_data * doc: improve documentation * doc: add release note * feat: enable scenario support through file parameter * doc: add explicit description of the preparation stage * fix: enable custom costs by default * fix: fix units case * doc: improve description of default values * fix: resolve year wildcard ambiguity * refactor: adjust naming convention * replace year with planning_horizons * remove expanding to all & overwriting only given costs without priotization * update configtable accordingly * Update rules/collect.smk Co-authored-by: Johannes HAMPP <42553970+euronion@users.noreply.github.com> * year bugfix * remove .format() in rules * harmonize costs and tech_costs * untangle params and config * dtype change * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * typo * Update config/config.default.yaml Co-authored-by: Johannes HAMPP <42553970+euronion@users.noreply.github.com> * Update doc/configtables/costs.csv Co-authored-by: Johannes HAMPP <42553970+euronion@users.noreply.github.com> * Update rules/build_electricity.smk Co-authored-by: Johannes HAMPP <42553970+euronion@users.noreply.github.com> * rename build_cost_data to process_cost_data * rename build_cost_data to process_cost_data ii * clean snakemake inputs * omit conig usage * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * calculate nyears from actual snapshots * from manual to default nyears calculation * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update doc/release_notes.rst Co-authored-by: Thomas Gilon <thomas.gilon@openenergytransition.org> * omit year as a wildcard * add 'all'-tech feature from Thomas * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * explicit mention of index_cols * Apply suggestions from code review Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de> --------- Co-authored-by: Martha Frysztacki <martha.frysztacki@gmail.com> Co-authored-by: Johannes HAMPP <42553970+euronion@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
* code: Add empty folders to repo * Update purge rule --------- Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
* Implement OETC integration * Refine docs
…n names (#1884) * Rename index in GeoDataFrame for clarity and more generic shape column names * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…valid setups (#1882) * update scaling factor for industrial demands with snapshot_weightings * remove multiplication with sn weightings, raise error instead for invalid configurations * Update scripts/time_aggregation.py --------- Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
* fix: fix wildcard definition for process_costs * chore: rename rule in mock_snakemake * doc: format doctstring * fix: reintroduce a two stage overwrite * doc: update release note * refactor: simplify overwrite_costs and improve clarity of code * fix: make custom costs filter resilient to missing columns * Apply suggestions from code review * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat: remove heat pump nominal efficiency and reallocate link buses and efficiencies * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update scripts/prepare_sector_network.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update scripts/prepare_sector_network.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update scripts/prepare_sector_network.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update scripts/prepare_sector_network.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * doc:update release notes * fix: adjust efficiency calculations in add_heat function to prevent division by zero * fix: remove redundant parameters in add_heat function * fix: increase non-zero COP limits to avoid small matrix coefficients * fix: correct bus parameter order and adjust efficiency calculation in add_heating_capacities_installed_before_baseyear function * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: correct cop clipping * feat: improve cop clipping * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: set default power limits in add_heating_capacities_installed_before_baseyear function * fix: correct capital cost calculation in add_heating_capacities_installed_before_baseyear function * fix: update p_min_pu calculation in heating capacity to prevent operation at COP=0 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: update efficiency calculations in add_heat function to handle edge cases * fix: adjust efficiency calculation in add_heat function to handle edge cases * fix: use negative efficiencies instead of negative p_min_pu * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * feat:replace clip_p_max_pu with rounding * cycle back to positive efficiencies, negative p * fix typo * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Revert "feat:replace clip_p_max_pu with rounding" This reverts commit d2a47a0. * fix: use absolute value for clipping p_min_max * fix multi-link heat pumps * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix efficiency2 in multi-link HPs * fix typo * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix heat pump capacity limits in add_heat function * remove unnecessary replace statements from COP --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* doc: Automatically update DAGs in documentation * doc: Update release notes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * I find working with the CI always tricky * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * some adjustments --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: lkstrp <lkstrp@pm.me>
The old expression ('transmission_expansion') is no longer in the config file and therefore fails to overwrite the 'transmission_limit' if it's used in the 'opts'
* code: Fix deprecation warning for pandas 2.3.0 / 3.0 * Update relase notes
…1896) * code: Fix PyPSA v1 compatability * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Update dependency management to `pixi` * Pre-commit fixes * Update docs deps to all latest-up * Revert validate.yaml; update reset task; unfreeze test env * Update install instructions; introduce conda pin files * Update dependencies; fix script dir PATH in docs build * Update PR template * Reintroduce envs/environment.yaml * Expand installation docs; expunge conda refs; update dockerfile * Update pixi version in test * Use latest pixi version throughout * Decouple dag build from docs build
…#1904) Bumps the github-actions group with 1 update: [prefix-dev/setup-pixi](https://github.com/prefix-dev/setup-pixi). Updates `prefix-dev/setup-pixi` from 0.9.1 to 0.9.3 - [Release notes](https://github.com/prefix-dev/setup-pixi/releases) - [Commits](prefix-dev/setup-pixi@v0.9.1...v0.9.3) --- updated-dependencies: - dependency-name: prefix-dev/setup-pixi dependency-version: 0.9.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Fix clean-tests section in pixi.toml * code: Always run cleanup tasks
* refactor: remove hotfix and check that the directory exists * doc: add release note * refactor: use streaming copy instead of loading into memory --------- Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
…ures (#1913) * feat: Add snakemake-storage-plugin-cached-http to alleviate zenodo failures * Add release note * fix: don't use progress bar for cost retrieval * Desperate debugging * More desperation * fix: fix progress_retrieve use iter_content to avoid storing transparently compressed data. * chore: clean up desperation * Apply suggestion from @lkstrp --------- Co-authored-by: Lukas Trippe <lkstrp@pm.me>
* Update locked environment files for all platforms * trigger * trigger --------- Co-authored-by: lkstrp <62255395+lkstrp@users.noreply.github.com> Co-authored-by: Lukas Trippe <lkstrp@pm.me>
* Update locked environment files for all platforms * Add trigger section to pixi.lock * Remove Trigger section from pixi.lock --------- Co-authored-by: lkstrp <62255395+lkstrp@users.noreply.github.com> Co-authored-by: Lukas Trippe <lkstrp@pm.me>
* Ensure cop_heat_pump always Pandas DataFrame * Reduce dataframe back to scalar after clipping. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add release note. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* initial dsm implementation for the heating sector * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * assume thermal losses in buildings and remove heating dsm from default * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * improve heat dsr documentation * fix for heat_dsm_profile for leap year (#1534) * fix for heat_dsm_profile for leap year * add fneum code suggestion * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * CI related * working workflow * working workflow * adding documentation * Fixing CI and adding impact assessment * addressing comments * deleting report * Fixing merge and addressing comments * support different dsm potentials for different planning horizons * config settings * improve and adjust documentation * Larger example * higher temporal resolution * comparing heat dsm vs no heat dsm * Fix docs: heat DSM restriction_time is timezone-adjusted * code: Restructure feature Rename config and restructure config. Add directionality option. * Fix colours for plotting * code: Consistent labelling of heat dsm * Update tests * minor typo * addressing comments * doc: Fix time description * doc: Update UTC/local time confusion * code: Fix linting complaints * pre-commit * Fix test config --------- Co-authored-by: martacki <martha.frysztacki@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Rüdt <117752024+daniel-rdt@users.noreply.github.com> Co-authored-by: euronion <johannes.hampp@openenergytransition.org> Co-authored-by: Johannes HAMPP <42553970+euronion@users.noreply.github.com>
* code: Versioning for hotmaps dataset * code: Fix CLI not working for foreign owned records * code: versioning for co2stop dataset * code: versioning for nitrogen statistics datset * code: fix wrong file name for REUSE.toml * code: versioning for NUTS 2013 and NUTS 2021 * code: Add limited debug mode for CLI script * code: Fix issues with datasets not being detected * code: Fix minor bugs on CLI and outputs * code: Versioning for eurostat energy balances * code: Remove obsolete retrieve script, now part of retrieve.smk * code: Versioning for eurostat household energy balances * code: Fix missing replacements for eurostat energy balances * code: Remove prompt for API key * code: Fix typo for enspreso biomass dataset * code: Fix issue with OSM dataset versioning * code: Versioning for WDPA and WDPA marine * code: Implement versioning for TYNDP base network * code: Fix bug missing osm_version in base_network.py * code: Properly fix the bug this time * code: Implement versioning for powerplants / powerplantsmatching * code/docs: Implement versioning for costs * code: Implement versioning for cutouts, move to data/ , deprecate shared_cutouts * Fix: Wrong URL for powerplants * code: Implement versioning for HDD and runoff per country (retrieval only) * code: Add build options for HDD and runoff per country (untested) * code: Add option for monthly retrieval of cutouts while building * code: Restore build_natura_raster.py script * code: Implement versioning for natura raster + reintegrate build natura raster script * data: Change URB POP dataset to Zenodo link * code: Adjust link for GEM GSPT * code: Versioning for GEM Europe Gas Tracker * doc: Correct license attributions in REUSE.toml for data/ * code: Implement versioning for BFS vehicle data * code: Implement versioning for BFS GDP and POP data * doc: Update data_inventory.csv for all the datasets added * code: Fix versions for HDD and runoff per country * code: Fix rules for creating HDD and runoff per country * code: Add special cutout config for HDD/runoff * code: Fix monthly_requests for cutouts * code: Add logic for building and versioning of mobility profiles from BASt * data: Update data for mobility profiles from BASt * code: Windows compatability for cutouts * code: Deprecate automatic cutout bounds * pre-commit formatting * code: Remove cutout dir configuration option * code: Fix option for archived mobility_profiles * code: Cleanup unused stub * config: Remove deprecated config option from configfiles * code: Remove prompt for API key * LUISA land cover database - data versioning * JRC-IDEES dataset versioning implementation * Remove obsolete script for retrieving jrc idees data * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scigrid gas network - versioning implementation * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * doc: scigrid dataset updates to versions.csv * Synthetic electricity demand versioning * Copernicus Global land service versioning * changing not-supported to might-work flags for some datasets * Ship raster - versioning implementation * EEZ dataset versioning implementation * NUTS3 population dataset versioning * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * GDP per capita dataset versioning * WorldPop population count dataset versioning * EEA ghg emissions data versioning * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * GEBCO versioning * Attributed Ports data versioning * Updating URLs * CORINE dataset versioning * Emobility dataset versioning * Common function to unzip folders * H2 salt caverns data versioning * Update primary link for GHG emissions * Primary link update for population count dataset * Lau regions data versioning * Data versioning for Aquifer data BGR * Updating archive link for lau regions * Retrieving corine dataset from primary datasource * OSM boundaries primary data versioning * OSM boundaries archive link update * rename function * code: Fix enspreso_biomass dataset name * code: Improve WDPA dataset handling * code: Simplify zip file handling * code: Simplify eurostat balances access * code: Centrally configure http storage provider * code: Fix typo * code: Enable config of prepare_kwargs * Bug fix - remove duplicate input entries for rule build_shapes * Removing check for retrieve flag * Moving corine Apikey from file to config parameter * Polishing the code * Code polishing for aquifer data set * code: Remove unnecessary code * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * code: simplify worldbank retrieval rule * change indexing method for rules * Remove code block for retrieving databundle * modifying the way to unzip folders * Code modifications * doc updates * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Script license * License correction * Doc formatting * code: Cleanup GEBCO retrieval * code: Always copy storage(...) retrieved data, manage keep_local centrally * code: remove params from retrieve_jrc_idees * code: remove params from retrieve_gas_infrastructure * code: remove params for retrieve_eez * code: Remove params for retrieve_osm_boundaries * code: Deprecate enable::retrieve, enable::retrieve_databundle, enable::retrieve_cost_data * code: addendum to centrally managing `keep_local` setting for storage(...) * code: formatting * doc: Update configtables for data:: config options * code: Cleanup old comments * doc: remove duplicate dataset * doc: Clarify `url` entry in data/versions.csv * Code cleanup * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding comments * Versioning for dh_areas * Data versioning for geothermal heat utilisation potentials * Update links for archive versions of dh_areas and geothermal heat utilisation potentials * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * doc: Configuration entries for dh_areas and geothermal_heat_utilisation_potentials * code: Remove obsolete databundle retrieval * doc: Update retrieve.rst * doc: Remove outdated information and sites * doc: Update PR template with hint to instruction on data versioning * code: Fix config["enable"]["retrieve"] artifacts * code: Update paths for accessing retrieved gas network * code: Remove obsolete script * code: Remove validate_checksum as it keeps failing from too many requests * code: Fix links for web archive versions * doc: Update formatting * doc: try different table formatting * code: Increase retries on retrieve * code: Fix in biomass potentials * code: Modify code to reference variable instead of index * code: Bug fix * code: correction to PyPSA set_option * remove conda in snakemake rule * code: Set max_requests_per_second * code: Adding versioning for JRC Ardeco dataset * code: JRC Ardeco URL updates * code: plot balance map bug fix * Update pypsa version to 0.33.2 * code: Cleanup ARDECO retrieval * code: Fix ARDECO retrieval rules * code: Adjust http storage limits to avoid Zenodo rate limit * debug: Extract information about Zenodo cert * Switch to primary sources for BFS data to avoid web archive retry limits * code: See if we can extract the Zenodo rate limit * code: Only request head * code: Align retrieve_corine with rest of code * code: Remove superflous parts * code: Add modified snakemake storage plugin * code: Fix typo * wip * WIP stuff * code: Bugfix workflow * nuclear approach to rate limiting errors * code: Fix EEZ retrieval from archive * feat: Add snakemake storage plugin for downloading zenodo files with ratelimit backoff, concurrent downloading and progress bars. * Update retrieve.smk to rely on zenodo storage plugin * Make pre-commit happy * fix url substring sanitization * Use tqdm-loggable to avoid flooding github logs * Add retries option to zenodo storage provider * fix: merge conflict to accomodate new rule build_dh_areas * fix lau_regions zenodo url * Re-add zenodo checksums (retrieved from api) and pull retrying into storage plugin * Replace aiohttp with httpx better connection pooling and timeout handling. * Release the stream a tiny bit earlier * fix: do cache data_versions for lru_cache to work the method needs to be at the toplevel. * Allow deactivating the cache and skipping all remote checks Use these settings for ci, since we already cache the resulting data folder. * Derive from settings from SettingsBase and clean up some docs * Revert "debug: Extract information about Zenodo cert" This reverts commit c1edb28d62c05c6a4efbd0c41d1e1e85e7fe4fc1. * fix: address unfounded fears * improve docstrings * feat: improve checksumming by allowing any digest * Update README.md * Expand user in `cache` directory * fix: missed fixes * cleanup * refactor: move storage plugin to own repo * refactor: rename `version_latest` to `version` * feat: allow passing corine secret via envvar * refactor: move zenodo cli to dedicated `utils` dir * Skip storage plugin cache in CI * feat: extract aquifer_data onaccess saves space. * fix: use vsizip to transparently decompress nuts3 2021 * fix: save synthetic electricity demand as csv.gz * fix: clean conda caches after update * Revert "fix: save synthetic electricity demand as csv.gz" This reverts commit 0ebc02c59b5348db48638929e579943ba2e62168. * Revert "fix: use vsizip to transparently decompress nuts3 2021" This reverts commit 9e6cd7f53612a2ff8a4e68cefe2b42884cebf70b. * Revert "feat: extract aquifer_data onaccess" This reverts commit 6de88a2257d0f022c378957ffd4fd784425e5f87. * remove tenacity and old checksum/ internet helpers * Adjust file paths to work on Windows Minor changes: - set paths to always use posix format * Fix file paths on windows nicely (#71) * fix: use string paths with / even on windows * feat: Add pixi toml and lock * fix: fix retrieve_worldbank_urban_population * Update rules/build_sector.smk * Revert "feat: Add pixi toml and lock" This reverts commit 9c9bf9748f53e89db33b1c97deb2774b3a69508c. * code: Remove quotation marks in config * Update comment * Update storage plugin config * code: Update pixi dependencies * code: Fix wildcard * Update rules/build_electricity.smk Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de> * Update doc/release_notes.rst Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de> * update `pixi.lock` * remove snakemake-storage-plugin-cached-http install from pypi * Revert "update `pixi.lock`" This reverts commit 6cecd76cdb9ad2de78864f939f728f30d04e7ac5. * revert updated lock file * fix pre-commit --------- Co-authored-by: Sermisha <sermisha.narayana@openenergytransition.org> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sermisha <156903227+SermishaNarayana@users.noreply.github.com> Co-authored-by: Lukas Trippe <lkstrp@pm.me> Co-authored-by: Jonas Hoersch <jonas.hoersch@openenergytransition.org> Co-authored-by: Jonas Hörsch <coroa@posteo.de> Co-authored-by: Eric Nitschke <eric.nitschke@tuhh.de> Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
* Fix BFS Swiss transport parsing * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add release notes --------- Co-authored-by: Danele Lerede <danelelerede@Martins-MacBook-Air.local> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* added existing biomass decentral and rural heating * added release notes * fixed typo * fixed typo in definitions * fixed pre-commit
* Fix 2.0 build_energy_totals for CH. * Updated release notes. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Set mock_snakemake to rule defaults. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…methanol (CC) (#1941)
Bumps the github-actions group with 1 update: [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request). Updates `peter-evans/create-pull-request` from 7 to 8 - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v7...v8) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-version: '8' dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* ci: run model runs only with src changes * add data
Updated warning and note sections for clarity and consistency.
…up (#1945) Bumps the github-actions group with 1 update: [actions/cache](https://github.com/actions/cache). Updates `actions/cache` from 4 to 5 - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](actions/cache@v4...v5) --- updated-dependencies: - dependency-name: actions/cache dependency-version: '5' dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the github-actions group with 2 updates: [actions/upload-artifact](https://github.com/actions/upload-artifact) and [actions/download-artifact](https://github.com/actions/download-artifact). Updates `actions/upload-artifact` from 5 to 6 - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](actions/upload-artifact@v5...v6) Updates `actions/download-artifact` from 6 to 7 - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@v6...v7) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions - dependency-name: actions/download-artifact dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…nd solving) (#1902) * refac: separate problem prep and solving * feat: add configs for rolling horizon and iterative cases of small BE model instance * move new rh and iterative configs to examples/ * docs: patch release notes * fix: enable brownfield + rh mode * refac: implement all feedback * remove RH configs --------- Co-authored-by: Lukas Trippe <lkstrp@pm.me>
* Fixes for network building from overpass (formerly osm-raw). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updated config.default and toplevel.csv configtable. * Cleaned up getters in retrieve_osm_data.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updated release notes. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
Co-authored-by: lkstrp <62255395+lkstrp@users.noreply.github.com>
* Renamed original static plotting to '_static' and changed all map outputs to 'RESULTS + maps/static/...'. Maps in resources are not touched. * Fix: Updated config_providers to read updated config key 'balance_map_static.' * Interactive map implementation and static map plotting refactoring. * removed H2 vmin and vmax. * Dropped enable toggle. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * removed test config. * Restore build_energy_totals from master * Reverted to old plotting keys for balance_maps and changed balance_map_static back to balance_map * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Reordered plotting.default.yaml and added configtables for interactive maps. * Added release notes. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
…costs:horizon functionality
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Before asking for a review for this PR make sure to complete the following checklist:
ariadne_allcompletes without errorsexport_ariadne_variableshas been adapted to the changesChangelog.mdmainhas been merged into the PRYYYYMMDDdescriptive_title