There should still be a very easy way to use a specific commit of the C core. I regularly update the submodule in this repo to test changes I make to the C core. Can we make it so that this functionality is not lost, or not made too complicated to use?

Furthermore, it is important to point to a specific version of the C core. We cannot guarantee compatibility with more than a single specific C core version due to experimental functions. For this reason, I don't think it's a good idea to stop vendoring the C core. Making it easier to link to an external C core is good, but that doesn't have to involve removing the vendored version.

While experimental functions cause some difficulties with linking to an external C core, they are important (I would say essential) to maintaining the quality of igraph while keeping our workload manageable.

I should also link here: igraph/igraph#2643

Yes, it is easy to link to a specific commit or version when building wheels. I now included a specific build script, but this can always be done in one way or another in cibuildwheels.

However, I think it sort of defeats the purpose of not allowing any other version when linking to an external version. Why then bother at all? I understand your desire to include experimental functions (although I disagree it is essential), but if this would prevent linking to any other external library, I'm not sure if this would work. Couldn't we make those experimental functions optional, and simply throw an "unimplemented" exception?

I am not asking to make it impossible to link to a different C core. I am asking not to break the current setup. This needs to be preserved:

• I need a way to push a simple commit to this repo and change which specific C core commit the py-igraph CI tests are run with.
• We need to make it very easy for people to link to the officially supported C core version, meaning that its sources should be vendored, just like before. This does not mean that linking to an external version is prevented.

As for experimental functions: as the person working the most on the C core, I maintain that this setup is essential for me to be able to continue my work. Your comments indicates that we are not on the same page about what experimental functions are and why they are necessary. It's best to clarify this in person at the next meeting.

With all due respect, I don't think that saying "I fix most bugs therefore things cannot change" is a valid argument.

This is actually a great example of a problem that has plagued igraph for a long time: we are too afraid to break anything that we end up stifling innovation to a trickle. In my view, and that is as important as yours @szhorvat , experimental functions are definitely optional. That might make us reconsider a few things and that's good: I don't think having functions that are essential but also probably broken is a great practice.

I am making it clear what I need to continue working on this project, which I am actually doing, but only for as long as the conditions make it possible. You did not respond to several requests for feedback in the past few months on any communication channel (here, chat or email), @iosonofabio, nor did you keep up with development, yet now you make an uncalled for attack, clearly without an understanding of the issue at hand. That is anything but "respect".

This is a PR about a specific topic, and I feel the discussion is going off the rails.

To stay on topic, which is not intended as an attack of any sort, having experimental functions as optional seems like a pretty innocuous change and I'm simply throwing my support behind it.

Happy to zoom or Whatsapp if you feel personally attacked and we can try to navigate through that feeling together.

My two cents to the original issue raised by @vtraag: back in the old days, maybe some time around python-igraph 0.6 or before, the C core was not vendored and I was constantly struggling with issues stemming from the fact that I or someone else tried to compile python-igraph and ended up linking to the wrong version of the C core (or, even worse, linking to the right, locally-installed version dynamically at compile time, only to find out that the linker picks a globally-installed version at runtime when someone types import igraph at the Python prompt). That was when I abandoned the idea of linking to an external igraph completely, at least with the official releases and with the default setup of the repository. Truth to be told, cibuildwheel was nowhere around that time (not even Github Actions), and people needed to compile python-igraph much more frequently than nowadays.

I still need to look at the PR in more detail to judge to what extent this is a step back to the old days and what the consequences would be.

Some more thoughts:

• Python projects are adopting the newer, declarative pyproject.toml file instead of a setup.py script, and I'm not sure for how long setup.py will remain supported in the Python ecosystem anyway, so extracting the build logic of the C core from setup.py to an external script that can possibly be invoked from a custom PEP 517 build backend is probably a good general direction.
• On the other hand, relying on two separate scripts (a batch file for Windows and a shell script for non-Windows systems) feels like unnecessary code duplication for me, especially taking into account that this is a Python project so you could just implement the build logic in Python and then it's mostly going to be platform-independent, apart from maybe a few branches that are different on Windows and on non-Windows systems.
• Right now each revision of the Python interface repository clearly states which C core revision it is guaranteed to be compatible with by the virtue of the git submodule system. By removing the submodule, we lose this benefit and in my experience there will be support requests where we will have to deal with people who try to link to the wrong revision of the C core (and it's unclear which is the correct revision of the C core, especially now that the develop and master branches have diverged quite a bit).
• The problem of picking the right version of the C core could be alleviated somewhat after reaching 1.0 with the C core - after all, the API should be stable after 1.0 so it should be safe to link to newer versions from master, apart from the problem caused by experimental functions. This could be solved with a compile-time macro for the Python interface where the user could declare whether they want wrappers for experimental functions, the default being false. Note that picking the right version of the C core will also be a problem for the new, ctypes-based Python interface as well so we'll need to figure this out sooner or later anyway.
• Right now this PR is a bit too large to digest for me. I'd try to break it into smaller chunks. First, I would maybe create a PR that simply extracts the build logic specific to the C core from setup.py to a separate Python script in scripts/, while leaving everything else intact. IMHO this would definitely be a step in the right direction when it comes to reducing bloat in setup.py and it could become a building block for a PEP 517 build backend later -- if we decide to go down that way instead of focusing on the new, ctypes-based interface. Once the build logic of the C core is separated, we can keep on thinking about whether it would make sense to completely remove the C core submodule from the repo, and/or to find alternative means for the Python interface to indicate which C core revision it is guaranteed to be compatible with.

@iosonofabio That is a good suggestion. Let us do the WhatsApp call. I would like to do it in late August when I am back from travelling, if that works for you.