Extra (fixed) arguments for tool calls

[covitof]

Hello everyone,

while using instructor for my projects, I came across a case where the tool call might need some (fixed) arguments which are not present in the actual chat, but are related to the more general context in which the chat takes place. For example: a bot used to register the user for some service might not know the user ID, and neither does the user. This information is available in the backend, though. In this case, a simple solution would be to simply prepend the user's prompt with their ID, but this is not a general solution (+ we might not want to copy paste everything into the chat for privacy reasons).

I worked out a solution myself by allowing an "extra_args" argument in from_response and a corresponding exclude list of parameters to exclude from the openai_schema, since we don't want the LLM to come up with these "fixed" arguments itself.

The actual implementation is very simple and requires minimal changes, but I don't know if this is a problem more people encountered and is worth solving. What do you think? I'd be happy to contribute in case!

[covitof]

p.s. I have just noticed the openai_function decorator was removed. Does this mean that we are "forced" to use the pydantic models for simple functions, too? This probably strongly affects the answer to my previous idea/question

[jxnl]

yeah i think its best to do a pydantic model and then doing a __call__ method.

I would use the validation_context to hydrate some additional information. or do something like

class Func(BaseModel):
   arg1:  any
   arg2: any
   
   def __call__(self, user_id: str):
       return f(self.arg1, self.arg2, user_id)

[covitof]

not sure this solves my problem though. The neat thing about the openai_function decorator was that I could then treat both pydantic models and functions in the same way, and handle function calls in one go, without having to "if-else" each case for each model. Not sure whether this is possible now.

Though if the discussion could be interesting for more people, I'd move this post somewhere else and continue! I'd for sure be happy to.

[jxnl]

can you share a code snippet? I removed the function decorator cause i wanted to have 'one way of doing something'

[covitof]

After playing a bit with your ideas, I think I can still achieve the same result as before. Basically the idea was that in my conversation handling package I could simply do

result = available_functions[function_name].from_response(response)

and treat everything on the same footing. This was neat because in my application I have both data structures and functions acting on these. Now if I "transform" the simple functions into pydantic models and overload the __call__, I'd have to have an extra result() to actually extract the result. obviously it's not a problem to add the extra check. And I understand the reason to "do it one way only"!

The idea about the validation_context works great.. thanks!