Use sysconfigdata and `build-details.json` from shared library build #842

mgorny · 2026-01-15T15:28:29Z

Checklist

Used a personal fork of the feedstock to propose changes
Bumped the build number (if the version is unchanged)
Reset the build number to 0 (if the version changed)
Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
Ensured the license file is being packaged.

Use the sysconfigdata from the shared library build rather than the static one. The main difference is that this ensures that programs link to the shared Python library rather than the static library. Other differences are mostly superficial (build directories, test invocation commands).

Also use the build-details.json from there. Unlike the static build version, it has all the data, including static and shared library, and extension build details.

Fixes #565

conda-forge-admin · 2026-01-15T15:30:09Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe/meta.yaml:

ℹ️ The recipe is not parsable by parser conda-souschef (grayskull). This parser is not currently used by conda-forge, but may be in the future. We are collecting information to see which recipes are compatible with grayskull.
ℹ️ The recipe is not parsable by parser conda-recipe-manager. The recipe can only be automatically migrated to the new v1 format if it is parseable by conda-recipe-manager.

_{This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/21262143921. Examine the logs at this URL for more detail.}

mgorny · 2026-01-16T18:01:10Z

Okay, so:

Looks like my change is also causing python3-config --ldflags --embed to link against the dynamic library, which seems reasonable. However, tests need updating for that.
Given Move libpython.so to another output #843, perhaps instead of trying to force shared/static from a single test, I'll just split the libraries and test each one separately.

mgorny · 2026-01-16T18:13:40Z

Oh wait, that's for 3.15+ only.

mgorny · 2026-01-16T19:52:03Z

Ok, it's green now.

rgommers · 2026-01-16T19:54:44Z

recipe/meta.yaml

      requires:
        - {{ stdlib('c') }}
        - {{ compiler('c') }}
+        - ripgrep


leftover from debugging?

No, it's actually used in tests. However, it was listed only for python output and was missing for libpython-static.

isuruf

This is gonna make recipes like uWSGI link to the shared python library instead of the static library from a build number bump.

mgorny · 2026-01-17T09:34:03Z

This is gonna make recipes like uWSGI link to the shared python library instead of the static library from a build number bump.

What's your suggestion?

isuruf · 2026-01-20T20:52:28Z

We can merge this to the dev branch along with a solution to #843 and make it the default for 3.15.

rgommers · 2026-01-21T11:13:09Z

@isuruf if the problem can only be fixed properly for 3.15, can build-details.json at least be patched up? It's currently just broken (e.g., https://peps.python.org/pep-0739/#libpython-link-extensions is missing); it's new in Python 3.14 and quite useful when working on cross-compilation. And surely that would be fully backwards compatible (I mean, no project in conda-forge like uWSGI will be relying on it).

mgorny · 2026-01-21T12:31:22Z

How about using an explicit migrator to switch all versions? I suppose that would involve a lot of rebuilding, a large part of it not strictly necessary.

Use the sysconfigdata from the shared library build rather than the static one. The main difference is that this ensures that programs link to the shared Python library rather than the static library. Other differences are mostly superficial (build directories, test invocation commands). Fixes conda-forge#565 Signed-off-by: Michał Górny <mgorny@quansight.com>

Signed-off-by: Michał Górny <mgorny@quansight.com>

Now python3-config defaults to shared linking, so perform that part of the test first. Then remove the shared libraries to force static linking. Signed-off-by: Michał Górny <mgorny@quansight.com>

…6.01.20.09.33.33 Other tools: - conda-build 25.11.1 - rattler-build 0.55.0 - rattler-build-conda-compat 1.4.10

isuruf · 2026-01-21T15:53:42Z

@isuruf if the problem can only be fixed properly for 3.15, can build-details.json at least be patched up?

Sure. That's okay.

How about using an explicit migrator to switch all versions?

I'm not sure what you mean by here. Migrator for all downstream python packages?

mgorny · 2026-01-21T15:57:32Z

How about using an explicit migrator to switch all versions?

I'm not sure what you mean by here. Migrator for all downstream python packages?

I'm not sure if I understand migrators correctly, but my thinking would be pinning to current python versions for all slots with "static build" default, and using a migrator to migrate to "shared build" default.

mgorny · 2026-01-21T15:58:12Z

Either way, rebased on dev to get the initial change in, and I'm working on libpython-devel now.

mgorny · 2026-01-21T17:32:22Z

(I'm assuming we still want python to install the minimum amount of development files required to build Python extensions, so pip install ... remains fully working.)

isuruf · 2026-01-21T17:38:58Z

Yes, we can move libpython.so to libpython and keep everything else in python as it is. Then users can install libpython or libpython-static if they want to embed python.

isuruf · 2026-01-21T17:41:16Z

I'm not sure if I understand migrators correctly, but my thinking would be pinning to current python versions for all slots with "static build" default, and using a migrator to migrate to "shared build" default.

We could, but there are many users of the python package outside of conda-forge, and I really don't want to break them. (There's definitely less users of libpython-static, but can't be sure of its users)

mgorny · 2026-01-21T17:42:30Z

Then users can install libpython or libpython-static if they want to embed python.

In #843, you said libpython-devel with both shared and static library. Am I following the wrong path?

isuruf · 2026-01-21T17:45:45Z

In #843, you said libpython-devel with both shared and static library. Am I following the wrong path?

We can't have the two together right? Looking at this PR and the tests, we have to manually delete the other if both exist.

mgorny · 2026-01-21T17:49:31Z

In #843, you said libpython-devel with both shared and static library. Am I following the wrong path?

We can't have the two together right? Looking at this PR and the tests, we have to manually delete the other if both exist.

Only for -lpythonX.Y to work with no extra effort. At least on Linux, -Bstatic and -Bdynamic let's you choose; I couldn't find anything for macOS. At least build-details.json has paths to both libraries, and I suspect CMake and Meson have their own lookup logic that handles both.

isuruf · 2026-01-21T17:52:45Z

Only for -lpythonX.Y to work with no extra effort.

People generally use python3-config --embed which should work properly.

mgorny · 2026-01-22T16:06:46Z

Looks like almost all failed on package output verification, with the exception of macOS that failed on cp options. I have a fix ready, but I'll wait with pushing till conda-forge/admin-requests#1854 is merged, so we could get clearer CI results.

mgorny · 2026-01-22T16:42:04Z

I've just noticed that I didn't do libpython*.so symlink correctly; working on fixing that now. Apparently the build directory doesn't use symlinks at all, and they're added in bininstall target.

Fixes conda-forge#843 Signed-off-by: Michał Górny <mgorny@quansight.com>

…6.01.22.10.05.09 Other tools: - conda-build 25.11.1 - rattler-build 0.55.0 - rattler-build-conda-compat 1.4.10

mgorny requested review from chrisburr, isuruf, jakirkham, katietz, mbargull, msarahan, ocefpaf, pelson, scopatz and xhochy as code owners January 15, 2026 15:28

mgorny force-pushed the ldlibrary-so branch from 5f5d56a to 5953d4c Compare January 16, 2026 18:22

rgommers reviewed Jan 16, 2026

View reviewed changes

isuruf requested changes Jan 16, 2026

View reviewed changes

mgorny added 5 commits January 21, 2026 16:21

Also use shared-lib build-details.json

dc22d85

Signed-off-by: Michał Górny <mgorny@quansight.com>

Add missing ripgrep dep to the test case

0e83970

Signed-off-by: Michał Górny <mgorny@quansight.com>

Fix the linking test case

b422d66

Now python3-config defaults to shared linking, so perform that part of the test first. Then remove the shared libraries to force static linking. Signed-off-by: Michał Górny <mgorny@quansight.com>

MNT: Re-rendered with conda-smithy 3.54.1 and conda-forge-pinning 202…

622caf6

…6.01.20.09.33.33 Other tools: - conda-build 25.11.1 - rattler-build 0.55.0 - rattler-build-conda-compat 1.4.10

mgorny changed the base branch from main to dev January 21, 2026 15:47

mgorny force-pushed the ldlibrary-so branch from 5953d4c to 622caf6 Compare January 21, 2026 15:51

isuruf mentioned this pull request Jan 21, 2026

Move libpython.so to another output #843

Open

mgorny force-pushed the ldlibrary-so branch from a176a7b to b6e118b Compare January 22, 2026 18:53

Split shared Python library to libpython

6eeda71

Fixes conda-forge#843 Signed-off-by: Michał Górny <mgorny@quansight.com>

mgorny force-pushed the ldlibrary-so branch from b6e118b to 5a2e6fa Compare January 22, 2026 18:55

MNT: Re-rendered with conda-smithy 3.54.1 and conda-forge-pinning 202…

a407c5d

…6.01.22.10.05.09 Other tools: - conda-build 25.11.1 - rattler-build 0.55.0 - rattler-build-conda-compat 1.4.10

mgorny force-pushed the ldlibrary-so branch from 5a2e6fa to a407c5d Compare January 22, 2026 19:32

Uh oh!

Use sysconfigdata and build-details.json from shared library build #842

Are you sure you want to change the base?

Use sysconfigdata and build-details.json from shared library build #842

Conversation

mgorny commented Jan 15, 2026

Uh oh!

conda-forge-admin commented Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mgorny commented Jan 16, 2026

Uh oh!

mgorny commented Jan 16, 2026

Uh oh!

mgorny commented Jan 16, 2026

Uh oh!

rgommers Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

mgorny Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

isuruf left a comment

Choose a reason for hiding this comment

Uh oh!

mgorny commented Jan 17, 2026

Uh oh!

isuruf commented Jan 20, 2026

Uh oh!

rgommers commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mgorny commented Jan 21, 2026

Uh oh!

isuruf commented Jan 21, 2026

Uh oh!

mgorny commented Jan 21, 2026

Uh oh!

mgorny commented Jan 21, 2026

Uh oh!

mgorny commented Jan 21, 2026

Uh oh!

isuruf commented Jan 21, 2026

Uh oh!

isuruf commented Jan 21, 2026

Uh oh!

mgorny commented Jan 21, 2026

Uh oh!

isuruf commented Jan 21, 2026

Uh oh!

mgorny commented Jan 21, 2026

Uh oh!

isuruf commented Jan 21, 2026

Uh oh!

mgorny commented Jan 22, 2026

Uh oh!

mgorny commented Jan 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Use sysconfigdata and `build-details.json` from shared library build #842

Use sysconfigdata and `build-details.json` from shared library build #842

conda-forge-admin commented Jan 15, 2026 •

edited

Loading

mgorny Jan 16, 2026 •

edited

Loading

rgommers commented Jan 21, 2026 •

edited

Loading