How to create a service (AIServices) with memory and streaming responses? #801
-
I would like to use a service with memory and streaming.
This code does not work if the type of the model is StreamingChatLanguageModel. |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 1 reply
-
You should just use .streamingChatLanguageModel instead of .chatLanguageModel |
Beta Was this translation helpful? Give feedback.
-
Hi @rgrin08, you need to define the return type of your AI Service method as |
Beta Was this translation helpful? Give feedback.
Hi @rgrin08, you need to define the return type of your AI Service method as
TokenStream
, please see an example here. You can add memory to it too.