Repository navigation
python-install fails because of stubgen: #167
Description
I am in a python3 conda environment, on MacOS 14.6.1,
(py310) (base) macbook-pro-8:build dellaert$ python --version
Python 3.10.14
Traceback (most recent call last):
File "/Users/dellaert/miniforge3/envs/py310/bin/pybind11-stubgen", line 8, in <module>
sys.exit(main())
File "/Users/dellaert/miniforge3/envs/py310/lib/python3.10/site-packages/pybind11_stubgen/__init__.py", line 319, in main
run(
File "/Users/dellaert/miniforge3/envs/py310/lib/python3.10/site-packages/pybind11_stubgen/__init__.py", line 358, in run
QualifiedName.from_str(module_name), importlib.import_module(module_name)
File "/Users/dellaert/miniforge3/envs/py310/lib/python3.10/importlib/__init__.py", line 126, in import_module
Traceback (most recent call last):
return _bootstrap._gcd_import(name[level:], package, level) File "/Users/dellaert/miniforge3/envs/py310/bin/pybind11-stubgen", line 8, in <module>
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
sys.exit(main())
File "/Users/dellaert/miniforge3/envs/py310/lib/python3.10/site-packages/pybind11_stubgen/__init__.py", line 319, in main
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
run(
File "/Users/dellaert/miniforge3/envs/py310/lib/python3.10/site-packages/pybind11_stubgen/__init__.py", line 358, in run
File "/Users/dellaert/git/github/build/python/gtsam_unstable/__init__.py", line 1, in <module>
QualifiedName.from_str(module_name), importlib.import_module(module_name)
File "/Users/dellaert/miniforge3/envs/py310/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
from .gtsam_unstable import *
ModuleNotFoundError: No module named 'gtsam_unstable.gtsam_unstable'
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/Users/dellaert/git/github/build/python/gtsam/__init__.py", line 9, in <module>
from gtsam import gtsam, utils
ImportError: cannot import name 'gtsam' from partially initialized module 'gtsam' (most likely due to a circular import) (/Users/dellaert/git/github/build/python/gtsam/__init__.py)
make[3]: *** [python/CMakeFiles/python-unstable-stubs] Error 1
make[3]: *** [python/CMakeFiles/python-stubs] Error 1
make[2]: *** [python/CMakeFiles/python-unstable-stubs.dir/all] Error 2
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [python/CMakeFiles/python-stubs.dir/all] Error 2
make[1]: *** [python/CMakeFiles/python-install.dir/rule] Error 2
make: *** [python-install] Error 2
Metadata
Metadata
Assignees
Labels
No labels
Activity
dellaert commentedon Sep 20, 2024
Actually, even after disabling the stub generation, the library installs fine but the same
circular importissue remains:Did something about this change in the last PRs? It's weird as it still works in CI. I remember that this circular thing was an issue a long time ago, but it's weird it pops up again now.
dellaert commentedon Sep 20, 2024
Seems the .so file was built for 3.12, but I made sure all cmake versions where 3.10... Does this ring a possible bell, @varunagrawal ?
varunagrawal commentedon Sep 20, 2024
Check the GTSAM_PYTHON_VERSION and Cmake variables for the Python version in general. This looks like it is being set to different things in different variables.
This issue has bitten me before as well but it's a simple manual change hence I never tried to fix it.
dellaert commentedon Sep 21, 2024
They are all 3.10
dellaert commentedon Sep 21, 2024
And, I did:
And it installs in right directory. But somehow something is generating 3.12 binaries?
dellaert commentedon Sep 21, 2024
It works in a conda 3.12 environment (again with custom cmake)
ProfFan commentedon Sep 21, 2024
@dellaert I assume you nuked the entire build folder right? Could you paste your
cmakelog here?dellaert commentedon Sep 21, 2024
Did you give it a try on Ubuntu with 3.10?
PS Should CI not test all python versions we support? It should also test that cmake works out of the box in a conda or virtualenv…
dellaert commentedon Sep 21, 2024
I could do a quick test with a devcontainer…
ProfFan commentedon Sep 22, 2024
I can test with a Docker in a sec
argpravardhan commentedon Jan 15, 2025
Hi. I think i am facing a similar issue. The compilation is going fine but during installation i see this
varunagrawal commentedon Mar 6, 2025
Did we manage to figure this out? @yambati03 also seemed to be bitten by this recently.
DLuminary commentedon Aug 24, 2026
That's an interpreter/ABI mismatch — stubgen was invoked as a bare pybind11-stubgen from PATH at the time, picking up conda's Python rather than the one CMake configured with. That was addressed in borglab/gtsam#2029 and borglab/gtsam#2545 by switching to ${PYTHON_EXECUTABLE} -m, so #167 is probably already fixed and just not closed.
Fix PYTHONPATH construction and generate stubs on Windows