Tool addition: Cloud Vision API

[daigo38]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• Pleas do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing?

My proposal is based on demands I encountered in a previous commissioned development. There are use cases such as extracting text information from non-text data (like images of handwritten nursing records) and creating derived documents (such as care planning documents) based on that information.

There is still a lot of non-text data in the world, and I believe that a tool specialized in transcription is necessary to harness the power of LLMs.

2. Describe the feature you'd like to see

I would like to add the Cloud Vision API as a tool.

The Vision API includes classic image recognition and has extremely high accuracy in character recognition compared to things like GPT vision, and there is demand in traditional enterprises.

I want to create a workflow that extracts character information from files uploaded by users and passes it to the LLM.

To achieve this, it seems we will also need a file upload function.

3. How will this feature improve your workflow or experience?

This feature will eliminate the need for each user to define their own tools, making the process more efficient and streamlined.

4. Additional context or comments

I can contribute to the development of this feature!

5. Can you help us with this feature?

• I am interested in contributing to this feature.

[guchenhe]

We're working on enabling file upload, a tool would be highly welcomed once that's released