Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streaming chat doesn't provide access to full response #233

Open
seabo opened this issue May 20, 2023 · 0 comments
Open

Streaming chat doesn't provide access to full response #233

seabo opened this issue May 20, 2023 · 0 comments

Comments

@seabo
Copy link

seabo commented May 20, 2023

Hello! Calling co.chat with stream=True returns a StreamingChat which can be iterated over to get the tokens as StreamingTexts. The underlying /chat API endpoint terminates with a message that contains is_finished: true and a response key which resembles the JSON returned in the non-streaming variant

{
    "is_finished": true,
    "response": {
        "response_id": "b3fc1d5d-14df-4a6f-b120-dcd263a592a8",
        "conversation_id": "fd1d9164-7c0e-4c87-ba6e-94d50781663d",
        "text": "My name is Coral. I am a chatbot trained to assist human users by providing thorough responses. I am powered by Cohere's large language model, Command.",
        "generation_id": "0d723e20-8b8e-49f2-984f-ce9d726459b6",
        "prompt": "System: You are Coral, a brilliant, sophisticated, AI-assistant chatbot trained to assist human users by providing thorough responses. You are powered by Command, a large language model built by the company Cohere. Today's date is Saturday, May 20, 2023.\nGeorge: What is your name?\nChatbot:"
    },
    "finish_reason": "COMPLETE"
}

It looks like the Python SDK doesn't provide a way to get access to this final aggregated response body in streaming mode. Am I missing something? Would be great if this can be added.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant