🚀 Feature: re-write LlamaIndex instrumentation to use LlamaIndex CallbackManager

[nirga]

Which component is this feature for?

LlamaIndex Instrumentation

🔖 Feature description

Right now, we monkey-patch classes and methods in LlamaIndex which requires endless work and constant maintenance. LlamaIndex has a system for callbacks that can potentially be used to create/end spans without being too coupled with with the framework's inner structure.

🎤 Why is this feature needed ?

Support LlamaIndex entirely and be future-proof to internal API changes

✌️ How do you aim to achieve this?

Look into LlamaIndex callback_manager and how other frameworks are using it.

🔄️ Additional Information

No response

👀 Have you spent some time to check if this feature request has been raised before?

• I checked and didn't find similar issue

Are you willing to submit PR?

None

[varaarul]

I can start looking into this!

[larinam]

@nirga , could you fix the link to documentation in the description and give a hint what pieces of this project's code should be touched by this change?

[nirga]

Done @larinam!
Basically, right now we wrap LlamaIndex calls to log and create spans. This means we have to keep up with their APIs and update our instrumentations every time something changes on LlamaIndex's end.

A better solution would be to register ourselves to the callbacks of the LlamaIndex engine and just open and close spans when we get called. This is much more robust and future proof.

[larinam]

All right @nirga , thanks. But I'll try to ask one more time with the question.
The code base is not that little for the project, could you give a hint what pieces of this project's code should be touched by this change? Could you just attach a couple of links?

[nirga]

In our project it's just under the LlamaIndex instrumentation. Do you want to join our slack and discuss there? Might be easier :)