You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi @andreibondarev, I noticed that the current version already supports sending images to LLMs.
You just need to include the image within the messages parameter. For example, when using OpenAI models, you can include images using the image_url content type. Here's how:
llm=Langchain::LLM::OpenAI.new(api_key: ENV["OPENAI_API_KEY"])llm.chat(messages: [{role: "user",content: [{type: "text",text: "What's in this image?"},{type: "image_url",image_url: {url: "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"}}]}],model: "gpt-4o").completion
Other LLMs only support sending the image in base64 format, but this must still be done within the messages parameter.
No description provided.
The text was updated successfully, but these errors were encountered: