Repository navigation
Xtb ignores ncores #355
Description
Describe the bug
when using xtb via QCEngine I notice that all available cores on my machine are being used despite passing a limit smaller than this to the compute function. This seems to be the default behaviour of xtb when OMP_NUM_THREADS or MKL_NUM_THREADS is not set in the environment. If I then set these variables both to 1 and then redo the calculation with ncores=4 xtb again ignores the qcengine input and instead respects the exported variable and runs on a single core. So it would seem that xtb is not picking up the variables set here.
I also wanted to check if this is how the resources should be set, could setting both of these to the ncore value lead to the use of more cores than allocated?
To Reproduce
A simple torsiondrive of ethane is attached using xtb-python, the files were renamed to make github happy.
ethane.txt
run_td.txt
Expected behavior
Xtb should respect the number of cores passed to ncores.
versions
Name Version Build Channel
qcengine 0.22.0 pyhd8ed1ab_0 conda-forge
xtb 6.4.1 h67747af_2 conda-forge
xtb-python 20.2 py38h96a0964_3 conda-forge
Activity
WardLT commentedon Mar 30, 2022
Thanks for reporting this. I've also seen this behavior before, and have some ways of working around it.
grimme-lab/xtb-python#65 and your experience shows that the OMP_NUM_THREADS should work. I'm just not sure what's going wrong with setting the environment variables, or if the environment context isn't being used by XTB.
Are you having any particular problem due to this bug? I'm figuring out how much we prioritize fixing this (e.g., if your blocked vs annoyed vs just observing an issue).
jthorton commentedon Mar 30, 2022
Thanks for the response and glad someone else has seen this I thought it was just the way I was using xtb at first.
It is more the annoying category right now having to remember to set
OMP_NUM_THREADSon each machine I run these calculations on otherwise we see a considerable slow down when it uses all cores.awvwgk commentedon Apr 20, 2022
The best choice would be to implement this in the
xtbC-API at https://github.com/grimme-lab/xtb and expose the option in the harness as described in the linked thread.mattwthompson commentedon Sep 19, 2022
Is there no shorter path to passing this through the Python layer?
awvwgk commentedon Sep 19, 2022
qcng might find a way to hack around the current limitation of the Python API of
xtb. Cleaner would be to support this option inxtband just set it from the harness. Either case requires a patch for qcng and maybe also forxtb.