bug: OpenAI 0.28 chat completion call gives response usage object that cannot be digested by langfuse

[Criss-Wang]

Describe the bug

Highly correlated to #1249 but the problem is specific to direct OpenAI API call. Obtained the same results from @istandleet. A potential fix was suggested by @gautamcrhythemx from that issue.

FYI: I've tried using the old release version as mentioned in that closed issue, but it givesImportError (incompatible with openai==0.28 I think)

from openai import AsyncAzureOpenAI, AsyncOpenAI, AzureOpenAI, OpenAI  # noqa: F401
ImportError: cannot import name 'AsyncAzureOpenAI' from 'openai'

Please kindly provide any suggestions if possible.

To reproduce

deps

openai==0.28.1
langfuse==0.27.1

code

import os

from langfuse.openai import openai
from langfuse.decorators import observe

openai.api_type = os.environ.get("OPENAI_API_TYPE")
openai.api_base = os.environ.get("OPENAI_API_BASE")
openai.api_version = os.environ.get("OPENAI_API_VERSION")
openai.api_key = os.environ.get("OPENAI_API_KEY")


@observe()
def run():
    prompt = [
        {"role": "system", "content": "You're a smart guy"},
        {"role": "user", "content": "What is OpenAI"},
    ]
    return openai.ChatCompletion.create(engine="GPT4", messages=prompt)


if __name__ == "__main__":
    run()

Error:

ValueError: Usage object must have either {input, output, total, unit} or {promptTokens, completionTokens, totalTokens}

Additional information

No response

[marcklingen]

What kind of exception is thrown at the latest OpenAI and Langfuse sdk versions?

[Criss-Wang]

@marcklingen This issue is specific to openai v0.x. The latest OpenAI sdk may have no issue with the call.

[marcklingen]

Thanks for confirming, as we are currently extremely busy working on new releases, I'd appreciate contributions to fix this issue for older openai versions. Is there a reason why you cannot upgrade openai?

[Criss-Wang]

Thanks for confirming, as we are currently extremely busy working on new releases, I'd appreciate contributions to fix this issue for older openai versions. Is there a reason why you cannot upgrade openai?

Yes, our app wants to provide backward compatibility as much as possible so we support both v1.x and v0.x. I'll try create a fix for this issue soon. Thanks for the update.