[Pytorch] Create CPP presets for other Pytorch versions

[carlosuc3m]

Hello I am developing an application that is able to load pre-trained Pytorch models and execute inference with them.
I want to try Javacpp to load Pytorch in Java. I have seen the Javacpp-presets for Pytorch but I assume that they are only available
for Pytorch 1.9.0. Is there any help I can use to try to create the presets for previous versions?
Thank you @saudet for pointing this repo in other forum.
Regards,
Carlos

[carlosuc3m]

I just found the documentation:
https://github.com/bytedeco/javacpp-presets/wiki/Create-New-Presets
Just out of curiosity, how does this approach allow loading and unloading of native libraries at runtime?

[saudet]

I'm not sure what you mean, "org.bytedeco.javacpp.nopointergc" can only be set once:
http://bytedeco.org/javacpp/apidocs/org/bytedeco/javacpp/Pointer.html#referenceQueue
If you plan on calling Pointer.interruptDeallocatorThread() though, we don't need to set "org.bytedeco.javacpp.nopointergc".
Yes, it works the same for other native libraries, as long as they support being unloaded.

[carlosuc3m]

Do I need to recompile the Pytorch Javacpp presets with the new JAvacpp version?

[saudet]

No, you can use the snapshots: http://bytedeco.org/builds/

[carlosuc3m]

Hello again @saudet .
Ive been trying the new presets and I am still finding some trouble. First of all when I load the pytorch presets with the following code:

try (URLClassLoader cls1 = new URLClassLoader(getURLs(file1))) {
    // Load native library of Pytorch
    Class<?> ca3 = cls1.loadClass("org.bytedeco.pytorch.Module");
    ca3.newInstance();
    Class<?> dealocateC = cls1.loadClass("org.bytedeco.javacpp.Pointer");
    Method dM = dealocateC.getMethod("interruptDeallocatorThread");
    dM.invoke(null);
    cls1.close();
    // REmove references and try to garbage collect
    ca3 = null;
}

I get the following warning:

OpenJDK 64-Bit Server VM warning: You have loaded library /home/carlos/.javacpp/cache/pytorch-1.10.1-1.5.7-20220104.184837-44-linux-x86_64.jar/org/bytedeco/pytorch/linux-x86_64/libtorch_cpu.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.

But then, when I tried to load another version of the libtorch_cpu.so just with a simple System.load() I get the following error:
java: ../nptl/pthread_mutex_lock.c:81: __pthread_mutex_lock: Assertion `mutex->__data.__owner == 0' failed.

I have to say that I have been able to load and unload the Pytorch dlls in Windows relatively easy but never in Linux.
What do you think?
Regards and thanks a lot for your help,
Carlos

[saudet]

Indeed, it doesn't appear to be possible to reliably unload libraries on Linux:
https://stackoverflow.com/questions/24467404/dlclose-doesnt-really-unload-shared-object-no-matter-how-many-times-it-is-call

[saudet]

It should work more or less automatically with 1.8.x. You'll need to change the version in the cppbuild.sh and pom.xml files, then after removing src/gen try to run mvn install on that, and you'll probably get a couple of compiler errors, but it should be all fixable by adjusting presets/torch.java a bit. Let me know if you encounter any errors that you are not able to resolve though.

[saudet]

Possibly, but not necessarily, there are some differences, not everything is always different, that's why there are versions!

[carlosuc3m]

Great! Thank you for all the help during the past few days! I might come back to ask a couple of questions :)
Regards,
CArlos

[saudet]

BTW, people usually do it the other way. They export models to a single new version of a runtime, which at the moment is centered around ONNX and ONNX Runtime, like those here from Hugging Face: https://huggingface.co/models

TVM is also an option, if you're willing to hack a bit more: http://bytedeco.org/news/2020/12/12/deploy-models-with-javacpp-and-tvm/

[carlosuc3m]

Yes I am familiar with that. However our application is oriented towards naive users so the version management should be done in the backend.

[carlosuc3m]

By the way Github actions cannot execute my build because it's longer than 6 hours, is there any way to alleviate it?