Cross compile linking issue: cannot locate symbol "_Py_NoneStruct"

[Progdrasil]

🐛 Bug Description

I need to cross compile a pyo3 library for android using kivy/buildozer. I've set the LD_LIBRARY_PATH, PYO3_CROSS_INCLUDE_DIR and PYO3_CROSS_LIB_DIR environment variables and am using cargo ndk for the build process and copying the .so to the site_packages directory.

Everything compiles correctly, unfortunately at runtime I get the message cannot locate symbol "_Py_NoneStruct".

🌍 Environment

• Your operating system and version: Ubuntu 20.04 (docker)
• Your python version: 3.8
• How did you install python (e.g. apt or pyenv)? Did you use a virtualenv?: compiled from source
• Your Rust version (rustc --version): 1.45.0
• Your PyO3 version: 0.11.1
• Have you tried using latest PyO3 master (replace version = "0.x.y" with git = "https://github.com/PyO3/pyo3")?: yes

💥 Reproducing

I have a repo for the minimal working example. The necessary commands are shown in the README.md

[davidhewitt]

Thanks for posting. This looks like a packaging issue, possibly in kivy/python-for-android. Here's an example of a very similar looking upstream bug from 2016 which was closed. kivy/python-for-android#887

As they will have much better knowledge than I for diagnosing the buildozer config, I suggest reporting upstream there before I attempt any debugging here.

[Progdrasil]

I'll check that out, however I did have a similar issue when compiling the same project in termux I'm not sure if this constitutes a different issue, but it is the same target.

ImportError: dlopen failed: cannot locate symbol "PyExc_BaseException" referenced by "/data/data/com.termux/files/home/pyo3-buildozer/target/debug/my_lib.so"

The target is aarch64-linux-android

[Progdrasil]

After reading through the comments it seems what has fixed it was adding the LDFLAG+="-lpythonX.Ym" flag. I did get a linking issue when removing the extension-module where the linking flags had -lpython without the version. So it could not find the appropriate python library.

The referenced comment

edit

I've managed to get it working with the following configuration:

• extension-module enabled
• added the rustflags -C link-args=-L<LD_LIBRARY_PATH> -lpythonX.Ym

And it ONLY works with the m version of the library. I tested this for targets aarch64-linux-android and armv7-linux-androideabi.

Since this is a linking issue with extension-module would it be related to #1040 ?

[davidhewitt]

Thanks for the update! Perhaps not #1040 but definitely worth remembering in this area. Let's keep this issue open until we adjust build scripts accordingly.

To figure out what's going on here - so it sounds to me like libpython.so does not exist in the distribution you're compiling for, but presumably libpython3.8m.so does?

[davidhewitt]

(I'm a bit confused because I thought the last python I'd seen where the library had the m suffix was 3.6, but I've never found documentation about what the suffix was for, so perhaps there's something else at play here...)

[programmerjake]

(I'm a bit confused because I thought the last python I'd seen where the library had the m suffix was 3.6, but I've never found documentation about what the suffix was for, so perhaps there's something else at play here...)

my guess is m is for minimal like in the python3-minimal debian package.

[davidhewitt]

Aha looks like it's showing if the ABI has pymalloc support. We can check it with sys.abiflags. However, it was definitely removed in 3.8: https://docs.python.org/3/whatsnew/3.8.html#build-and-c-api-changes

We could fix build scripts to check this for Python 3.5-3.7.

@Progdrasil, you're definitely cross compiling against Python 3.8 when you have this m suffix?!

[Progdrasil]

Thats really interesting because i am compiling against 3.8. and both the 'm' version and normal versions are available. Ive tried linking against both and only the 'm' version works at runtime. If i link against the normal version i get the symbol not found error at runtime

I am curious to try compiling against 3.7 now. I'll update you when I have

[davidhewitt]

After exploring the python-for-android codebase it looks to me like they're generating this libpython3.8m.so name themselves, so I've opened an upstream patch.

There are probably still fixes we can introduce on Python 3.5-3.7 for this m suffix, where it is a valid identifier. EDIT: looks like we check sysconfig.get_config_var('LDVERSION') in build.rs, which (at least on my system) does include the m suffix for Python 3.7.

[Progdrasil]

looks like we check sysconfig.get_config_var('LDVERSION') in build.rs, which (at least on my system) does include the m suffix for Python 3.7.

Yes but that is used for local compilation, not cross compilation as I am doing. Instead you parse the header files and apply the major and minor version. Is there a way to get the configuration of the cross compiled version to see if it is being compiled as the m variant?

[davidhewitt]

Yes but that is used for local compilation, not cross compilation as I am doing. Instead you parse the header files and apply the major and minor version. Is there a way to get the configuration of the cross compiled version to see if it is being compiled as the m variant?

Indeed, you're correct and I was thinking about this this morning too. Actually I think that the cross compilation header parsing is both broken and brittle; I'm thinking about whether we need to try a solution using bindgen.

If I were to kick off a PR to use bindgen to detect cross-compile info, would you be willing to help test it? (And maybe even help finish it off, depending on how much time I can find for it...?)

[Progdrasil]

Also I succeeded in compiling for 3.7 and running the sysconfig.get_config_var(LDVERSION) on the target for 3.7 it returned 3.7m and when running with 3.8 it returned 3.8. However I think your merge request helps because the issue on the target was the fact that I was linking against libpython3.8.so while the only object that was copied to the target was libpython3.8m.so.

[Progdrasil]

Yeah I would be glad to help out! I can also help contribute to it as well

[davidhewitt]

Great! I'll try to either get notes or a rough implementation written up tomorrow or soon after.