TensorFlow with Google Coral.ai

[grasponcrypto]

Wondering if there is a way to utilize the coral.ai Edge TPU for facial/object recognition.

[alextran1502]

I am working on both features for local hosted services. Once that done, I will be working on an option for external services to be used for those machine learning services since a lot of people has their own model trained already 😃 I will keep your suggestion in mind

[Schluggi]

Any updates? :)

[mertalev]

The model engine we use, ONNX Runtime, doesn't currently support Coral (relevant issue). I'm happy to add it if/when this becomes an option.

[afrojewelz]

looking forward to see that outcome

[ptichalouf]

I'm here if U need some tester :)

[KillahB33]

Excited to see the transition to python for the ai piece which hopefully means that this will be coming soon enough!

[pdarcos]

I guess there is still no news on this from upstream?

[bo0tzz]

You would need to check the linked upstream issue for that.

[nikefate]

Feature Request for ONNX Runtime is closed.  Is there any possibility of obtaining support for Coral in another way?

[bo0tzz]

I think you're looking at the wrong upstream issue, it's still open.

[bo0tzz]

We investigated this again recently and the Coral is just not a capable enough device to run the models we use for Immich, so I'm closing this feature request.

[Timi7007]

ONNX seems to support Hailo modules, like the new Raspberry AI module, though. Would this be an option?

[bo0tzz]

The main problem with these small TPUs is that they just don't have anywhere near enough RAM for the models that Immich uses. I can't find any actual numbers for Hailo, but it seems like it's just as far off as the Coral.

[mertalev]

I think NPUs just use normal RAM like iGPUs, so it may be different here.

[bo0tzz]

Hard numbers don't seem to exist, but a comment on hackernews guesses based on the models listed that it's probably some tens of MBs.

[chelming]

Halio-8L can run TensorFlow, TensorFlow Lite, Keras, PyTorch & ONNX models, where the Coral only does TFlite (and might max out at 6mb model size?).

I don't see any reference to a maximum model size for the Halio-8L and whether it needs to load the whole model into its own memory. There's not a lot of documentation or spec information for them.

[mertalev]

They have a CLIP section in their examples (coming soon ™).