You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Let's run an automated evaluation on the RAG dataset (using a local model or something) and then tune the doc search hyperparameters based on this. Parameters are:
Let's run an automated evaluation on the RAG dataset (using a local model or something) and then tune the doc search hyperparameters based on this. Parameters are:
k_docs
docs_confidence
https://github.com/julep-ai/julep/blob/dev/agents-api/agents_api/models/entry/proc_mem_context.py#L13
rag dataset: rag-12000
contains three columns:
context
,question
,answer
evaluation recipe:
create an agent
add all the documents from the
context
column as agent docsfor every row in the dataset (use the train split only)
question
column (you can set max_tokens to 1 since we dont care about the returned answer)context
(value of that row) is in the fetched documentscool thing: optuna: https://optuna.org/
The text was updated successfully, but these errors were encountered: