Options for extracting data files from a `RustExtension`'s build-script `OUT_DIR`

[jakelishman]

I have a mixed Python/Rust project using setuptools-rust to build a Rust extension module. This extension module, amongst other things, provides a C API, which I want to expose to external Python extension modules, similarly to how Numpy does (and rust-numpy uses) with a function-pointer table stored in a PyCapsule. setuptools-rust already solves all my needs for generating the shared-object library extension module, and I can create the PyCapsule in PyO3 easily, etc. However, I want to generate the C header file dynamically during the Rust build (using cbindgen as a library), and have the result included in my Python project as a data file, so downstream extension modules can use my Python module as a build dependency.

I think a sensible way I could do this would be if RustExtension provides a way for me to specify that a certain list of files or directories should be installed from the OUT_DIR of the build script into a specified location within the Python package. I'm fine if this is limited to a static mapping in pyproject.toml.

I'd rather avoid introducing a new build system, or having a bootstrap process before sdist creation; I'm happy with setuptools-rust.

I'm happy to produce a prototype PR myself, if this is something that's in scope for the project.

[jakelishman]

A complication I hadn't anticipated: a RustExtension definition doesn't necessarily correspond one-to-one with a call to cargo build. The most obvious (and problematic) example is macOS universal2 wheels (which I never used, preferring separate Intel and ARM wheels), but the multiple-binary form of RustExtension.target has similar behaviour.

This makes it less clear to me how to design a sensible API around those concerns, since I was assuming that the OUT_DIR was unique for a single build.