Support Linking Execution Trace to DatasetItem without Fetching Entire Dataset

[marcklingen]

Discussed in https://github.com/orgs/langfuse/discussions/1862

^{Originally posted by sfarthin April 25, 2024}

Describe the feature or potential improvement

We need a method to efficiently link an execution trace to a specific DatasetItem without the need to fetch the entire dataset first. Currently, the process requires retrieving a full dataset just to access a single Dataset Item, which is not optimal for performance.

The Langfuse API currently supports creating a dataset run item and linking it with a traceId as described here: Langfuse API - Create Dataset Run Item. However, to use the link method from the SDK, one must first fetch an entire dataset to obtain a single Dataset Item.

Proposal:

Develop a feature or method within the SDK/API that allows linking a trace directly to a Dataset Item by ID, without the necessity of loading the entire dataset.

const datasetItem = await langfuseClient.getDatasetItem(datasetItem.id);
datasetItem.link(trace, 'some_run_name');

This feature would be particularly useful in scenarios where users are dealing with large datasets and need to perform operations on individual items without the overhead of loading entire datasets.

Additional information

No response

[sudhanshu746]

Hi @marcklingen , we are also in need of this feature. Thanks!