SDK v2/v3

[mmabrouk]

Here are several enhancement proposals for the next iteration of our SDK, aimed at increasing its usability and flexibility.

Firstly, we've identified certain limitations in the current version that hinder developer experience:

1. Incorporating multiple parameters in a single call often results in cumbersome function calls. Let's take an example where an application makes three calls to GPT-3.5 and wishes to expose each model's configuration. Currently, we end up with a lengthy function call like this:

@post
def generate(inputs:str, temperature_model1:float, temperature_model2:float.......)

A more streamlined approach would be:

@post
def generate(inputs:str):
    llm_call1(inputs)
    llm_call2(inputs)

@llm_call
def llm_call1(inputs, temperature1...)

@llm_call
def llm_call2(inputs,temperature2...)

With this design, all the parameters get added directly to the API and the playground. An even more simplistic and user-friendly approach might look like this:

import agenta as ag
@post
def generate(inputs:str):
    output=llm_call1(inputs)
    return llm_call2(output)

@llm_call
     def (inputs, default_prompt:ag.TextParam=default_prompt):
     return ag.completion(default_prompt.format(inputs))

@llm_call
    def (inputs, default_prompt:ag.TextParam=default_prompt):
    return ag.completion(default_prompt.format(inputs))

In this setup, we could use the @llm_call to track the latency of each function and the ag.completion call to track costs, tokens, and prompts. This will also allow us to display all the parameters needed to configure the LLM by default.

2. Flexibility in number of inputs:
   Users might wish to adjust the number of inputs in a single prompt application.

We could redesign the SDK such that the user code looks like this:

@post
def generate(inputs:ag.DictInput, TextParam=default_prompt):
    prompt = default_prompt.format(**inputs)
    output=llm_call1(prompt)
    return output

However, this poses a challenge. The configuration of expected inputs will now need to be stored in the backend instead of openapi.json. Essentially, when the frontend encounters an ag.Dict in our inputs, it should automatically add a new parameter (list of inputs) which it will 1) save in the backend but not use in the call (to make things simpler, we can add that parameter to the fastapi call but not use it, thus maintaining the same interaction between the frontend and backend).