f32

[songololo]

#22

Makes use of f32 for reading, writing, internal arrays.

[biglimp]

Hi Gareth, can you please to clarify how these changes affect our code in UMEP for QGIS. As far as we can see, you have changed e.g. solweig_run.py. Have you tested this in QGIS environment? I think maybe we have missed this from a previous PR. Why are you making changes in dailyshading.py as this is not used in SOLWEIG? Please clarify?

[songololo]

Hi @biglimp - for this pull request I tried to change all instances of saving rasters to use f32 by default. I searched for all instances of common.save_raster lines where I added coerce_f64_to_f32=True which is a new setting to cast f64 arrays to f32 when saving. If preferred these can be set to False to leave as f64.

The changes to dailyshading.py were to the common.save_raster lines. When saving the file on my vscode it auto formats using the recommended ruff formatter, so the other changed lines are to do with formatting; e.g. ((vegdem2 * vegdem)) became (vegdem2 * vegdem) and int(0) became 0. It also automatically wraps or unwraps lines according to the formatting spec.

The changes to Solweig_run.py are similar (coerce_f64_to_f32=True).

What would be good is to find a way to test the common.py file using some automatic tests since this file needs to automatically detect whether it is running with access to rasterio or not, in which case we assume it is using gdal from QGIS. I'm not sure how to simulate a QGIS environment for testing but if anyone has ideas it would be nice to get some tests setup so that they automatically run when creating a pull request.

Another nice change would be to separate old code into a separate directory and to then auto format all current code using ruff which will help catch some potential errors like the f-string errors that pop up from time to time.

[biglimp]

Hi. Thanks for this info. I think you point towards the issue that we cannot control. There is no good way to test within a QGIS environment without doing it manually.

On another note, @nilswallenberg and I are discussing to create a separate pypi for the solweig model consisting of only the model itself, without any other workflow tools such as SVF-calculator. Then we can all use that pypi module to run the actual model. Do you see any issues with this?

[songololo]

@biglimp apologies for the slow response.

At the moment umep-core is designed to work either from Python or from QGIS (not tested at this point).

• SolweigRun - contains neutral logic
• SolweigRunCore - subclasses SolweigRun to implement vanilla Python logic
• SolweigRunQgis - subclasses SolweigRun to implement QGIS logic

For this to work, the common.py module is intended to check whether running in QGIS or not and to execute reading and writing operations accordingly (GDAL vs rasterio).

Personally I would recommend to keep SOLWEIG and other core functions for SVF and shadows together so that it isn't necessary to install and manage multiple packages. Though it would be easier to maintain if old or dead code is removed (or moved to a deprecated folder).

[biglimp]

@biglimp apologies for the slow response.

At the moment umep-core is designed to work either from Python or from QGIS (not tested at this point).

• SolweigRun - contains neutral logic
• SolweigRunCore - subclasses SolweigRun to implement vanilla Python logic
• SolweigRunQgis - subclasses SolweigRun to implement QGIS logic

For this to work, the common.py module is intended to check whether running in QGIS or not and to execute reading and writing operations accordingly (GDAL vs rasterio).

Personally I would recommend to keep SOLWEIG and other core functions for SVF and shadows together so that it isn't necessary to install and manage multiple packages. Though it would be easier to maintain if old or dead code is removed (or moved to a deprecated folder).

Sorry for late reply. I see your point but nevertheless, SVF and wall algorithm are separate tools that users make use of for many other applications a part from SOLWEIG-calculation. SOLWEIG is a separate model and therefore, it would be better to maintain this separately.

[biglimp]

Pulled the f32 branch and tried in the osgeo env. Got the following error:

C:\Users\xlinfr\Documents\PythonScripts\umep-core>C:/OSGeo4W/bin/python.exe c:/Users/xlinfr/Documents/PythonScripts/umep-core/demos/solweig_gbg_test.py
Traceback (most recent call last):
  File "C:\Users\xlinfr\Documents\PythonScripts\umep-core\umep\common.py", line 11, in <module>       
    import rasterio
  File "C:\OSGeo4W\apps\Python312\Lib\site-packages\rasterio\__init__.py", line 25, in <module>       
    from rasterio._base import DatasetBase
ImportError: DLL load failed while importing _base: The specified procedure could not be found.       

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\OSGeo4W\apps\Python312\Lib\site-packages\osgeo\__init__.py", line 30, in swig_import_helper
    return importlib.import_module(mname)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\OSGeo4W\apps\Python312\Lib\importlib\__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 921, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 813, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 1293, in create_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
ImportError: DLL load failed while importing _gdal: The specified procedure could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\Users\xlinfr\Documents\PythonScripts\umep-core\demos\solweig_gbg_test.py", line 4, in <module>
    from umep import (
  File "C:\Users\xlinfr\Documents\PythonScripts\umep-core\umep\skyviewfactor_algorithm.py", line 53, in <module>
    from umep import class_configs, common
  File "C:\Users\xlinfr\Documents\PythonScripts\umep-core\umep\class_configs.py", line 19, in <module>
    from . import common
  File "C:\Users\xlinfr\Documents\PythonScripts\umep-core\umep\common.py", line 21, in <module>
    from osgeo import gdal
  File "C:\OSGeo4W\apps\Python312\Lib\site-packages\osgeo\__init__.py", line 35, in <module>
    _gdal = swig_import_helper()
            ^^^^^^^^^^^^^^^^^^^^
  File "C:\OSGeo4W\apps\Python312\Lib\site-packages\osgeo\__init__.py", line 32, in swig_import_helper
    return importlib.import_module('_gdal')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\OSGeo4W\apps\Python312\Lib\importlib\__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named '_gdal'

C:\Users\xlinfr\Documents\PythonScripts\umep-core>

Here are the python packages installed (python 3.12.9):

C:\Users\xlinfr\Documents\PythonScripts\umep-core>pip list
Package                       Version
----------------------------- -------------
affine                        2.4.0
alabaster                     0.7.16
annotated-types               0.7.0
anyio                         4.3.0
appdirs                       1.4.4
argon2-cffi                   23.1.0
argon2-cffi-bindings          21.2.0
asteval                       1.0.0
asttokens                     2.4.1
async-lru                     2.0.4
atmosp                        0.2.9
attrs                         23.2.0
Babel                         2.14.0
beautifulsoup4                4.12.3
bleach                        6.1.0
cads-api-client               1.0.3
cdsapi                        0.7.6
certifi                       2024.2.2
cffi                          1.16.0
cftime                        1.6.4
chardet                       5.2.0
charset-normalizer            3.3.2
click                         8.1.7
click-plugins                 1.1.1
cligj                         0.7.2
cloudpickle                   3.0.0
colorama                      0.4.6
comm                          0.2.2
contourpy                     1.2.1
coverage                      7.4.4
crc32c                        2.7.1
cycler                        0.12.1
dask                          2024.6.2
datapi                        0.1.1
debugpy                       1.8.5
decorator                     5.1.1
defusedxml                    0.7.1
dill                          0.3.8
docutils                      0.20.1
donfig                        0.8.1.post1
ecmwf-datastores-client       0.1.0
et_xmlfile                    1.1.0
executing                     2.0.1
ExifRead                      3.0.0
f90nml                        1.4.4
f90wrap                       0.2.16
fastjsonschema                2.19.1
fiona                         1.10.1
flexcache                     0.3
flexparser                    0.3.1
fonttools                     4.51.0
fsspec                        2024.6.1
future                        1.0.0
GDAL                          3.10.2
geographiclib                 2.0
geopandas                     1.0.1
h11                           0.14.0
h3                            4.3.1
h5netcdf                      1.6.1
h5py                          3.11.0
httpcore                      1.0.5
httplib2                      0.22.0
httpx                         0.27.0
idna                          3.7
imagesize                     1.4.1
importlib_metadata            7.1.0
ipykernel                     6.29.5
ipython                       8.26.0
ipywidgets                    8.1.2
JayDeBeApi                    1.2.3
jedi                          0.19.1
Jinja2                        3.1.3
joblib                        1.4.0
JPype1                        1.5.0
json5                         0.9.25
jsonschema                    4.21.1
jsonschema-specifications     2023.12.1
jupyter                       1.1.1
jupyter_client                8.6.2
jupyter-console               6.6.3
jupyter_core                  5.7.2
jupyter-events                0.10.0
jupyter-lsp                   2.2.5
jupyter_server                2.14.0
jupyter_server_terminals      0.5.3
jupyterlab                    4.1.5
jupyterlab_pygments           0.3.0
jupyterlab_server             2.26.0
jupyterlab_widgets            3.0.10
kiwisolver                    1.4.5
linecache2                    1.0.0
llvmlite                      0.42.0
lmfit                         1.3.1
locket                        1.0.0
lxml                          5.3.0
markdown-it-py                4.0.0
MarkupSafe                    2.1.5
matplotlib                    3.10.0
matplotlib-inline             0.1.7
mdurl                         0.1.2
mistune                       3.0.2
mock                          5.1.0
multiprocess                  0.70.16
multiurl                      0.3.3
munch                         4.0.0
nbclient                      0.10.0
nbconvert                     7.16.3
nbformat                      5.10.4
nest-asyncio                  1.6.0
netCDF4                       1.7.1.post1
networkx                      3.3
nose                          1.3.7
nose2                         0.14.1
notebook                      7.1.2
notebook_shim                 0.2.4
numba                         0.59.0
numcodecs                     0.16.0
numdifftools                  0.9.41
numexpr                       2.10.1
numpy                         1.26.2
odfpy                         1.4.1
oldest-supported-numpy        2023.12.21
openpyxl                      3.1.2
overrides                     7.7.0
OWSLib                        0.32.0
packaging                     24.0
pandas                        2.2.2
pandocfilters                 1.5.1
parso                         0.8.4
partd                         1.4.2
pb-tool                       3.1.0
pillow                        11.1.0
Pint                          0.24.1
pip                           25.2
platformdirs                  4.2.2
Platypus-Opt                  1.0.4
plotly                        5.20.0
ply                           3.11
pooch                         1.8.2
prometheus_client             0.20.0
prompt_toolkit                3.0.47
psutil                        6.0.0
psycopg2                      2.9.10
pure_eval                     0.2.3
pvlib                         0.11.0
pyarrow                       22.0.0
pycparser                     2.22
pydantic                      2.11.7
pydantic_core                 2.33.2
Pygments                      2.17.2
pyodbc                        5.1.0
pyogrio                       0.9.0
PyOpenGL                      3.1.7
pyparsing                     3.1.2
PyPDF2                        3.0.1
pypiwin32                     223
pyproj                        3.7.0
PyQt5                         5.15.11
PyQt5_sip                     12.16.1
PyQtWebEngine                 5.15.7
pyserial                      3.5
python-dateutil               2.9.0.post0
python-json-logger            2.0.7
pytz                          2024.1
pywin32                       306
pywinpty                      2.0.13
PyYAML                        6.0.1
pyzmq                         26.2.0
qtconsole                     5.6.1
QtPy                          2.4.1
rasterio                      1.4.3
referencing                   0.34.0
remotior_sensus               0.4.4
reportlab                     4.2.5
requests                      2.31.0
retrying                      1.3.4
rfc3339_validator             0.1.4
rfc3986_validator             0.1.1
rich                          14.2.0
rioxarray                     0.18.2
rpds-py                       0.18.0
ruff                          0.12.8
scikit-learn                  1.4.2
scipy                         1.13.0
seaborn                       0.13.2
Send2Trash                    1.8.3
setuptools                    75.3.0
shapely                       2.0.6
simplejson                    3.19.2
sip                           6.9.1
six                           1.16.0
sniffio                       1.3.1
snowballstemmer               2.2.0
soupsieve                     2.5
Sphinx                        7.3.7
sphinx-rtd-theme              2.0.0
sphinxcontrib-applehelp       1.0.8
sphinxcontrib-devhelp         1.0.6
sphinxcontrib-htmlhelp        2.0.5
sphinxcontrib-jquery          4.1
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.7
sphinxcontrib-serializinghtml 1.1.10
stack-data                    0.6.3
supy                          2025.11.20rc1
target-py                     0.1.1
tenacity                      8.2.3
terminado                     0.18.1
threadpoolctl                 3.4.0
timezonefinder                6.5.9
tinycss2                      1.2.1
toml                          0.10.2
toolz                         0.12.1
tornado                       6.4.1
tqdm                          4.66.4
traceback2                    1.4.0
traitlets                     5.14.3
typing_extensions             4.14.0
typing-inspection             0.4.1
tzdata                        2025.2
tzfpy                         1.1.0
uncertainties                 3.2.1
unittest2                     1.1.0
urllib3                       2.2.1
uv                            0.6.8
uwg                           5.8.13
wcwidth                       0.2.13
webencodings                  0.5.1
websocket-client              1.7.0
widgetsnbextension            4.0.10
wxPython                      4.2.2
xarray                        2025.3.0
xlrd                          2.0.1
xlwt                          1.3.0
zarr                          3.0.8
zipp                          3.18.1

Am I missing something?

[biglimp]

Just made a simple test so that it was not my VSCode who did something strange. the code below:

import sys, os, rasterio
print("PYTHON:", sys.executable)
print("RASTERIO:", os.path.dirname(rasterio.__file__))
print("PATH:", os.environ["PATH"].split(";"))

gives below with no errors:

C:\Users\xlinfr\Documents\PythonScripts\umep-core>C:/OSGeo4W/bin/python.exe c:/Users/xlinfr/Documents/PythonScripts/umep-core/umep/importtest.py
PYTHON: C:\OSGeo4W\bin\python.exe
RASTERIO: C:\OSGeo4W\apps\Python312\Lib\site-packages\rasterio
PATH: ['C:\\OSGeo4W\\apps\\qt5\\bin', 'C:\\OSGeo4W\\apps\\Python312\\Scripts', 'C:\\OSGeo4W\\bin', 'C:\\windows\\system32', 'C:\\windows', 'C:\\windows\\system32\\WBem', 'C:\\OSGeo4W\\apps\\qgis\\bin', 'C:\\OSGeo4W\\apps\\Qt5\\bin', 'C:\\OSGeo4W\\apps\\Python312\\Scripts', 'c:\\Users\\xlinfr\\.vscode\\extensions\\ms-python.debugpy-2025.16.0-win32-x64\\bundled\\scripts\\noConfigScripts']