v0.14.6 - Ollama & News Updates

Chatbot Updates

• Expand /news/ RAG command to include reference URL links in news article headlines.
• Add response statistics (number of tokens and tokens per second) to footer.
• Serve up local copy of socket.io.js library to help with air-gap installations.

Ollama Support

• Add logic to chatbot to support OpenAI API servers that do not support the /v1/models API. This allows the Chatbot to work with Ollama provided the user specifies the LLM_MODEL, example docker run script:

docker run \
    -d \
    -p 5000:5000 \
    -e PORT=5000 \
    -e OPENAI_API_KEY="Asimov-3-Laws" \
    -e OPENAI_API_BASE="http://localhost:11434/v1" \
    -e LLM_MODEL="llama3" \
    -e USE_SYSTEM="false" \
    -e MAXTOKENS=4096 \
    -e TZ="America/Los_Angeles" \
    -v $PWD/.tinyllm:/app/.tinyllm \
    --name chatbot \
    --restart unless-stopped \
    jasonacox/chatbot

Full Changelog: v0.14.4...v0.14.6

v0.14.4 - Llama-3 Support

v0.14.4 - Llama-3 Support

• Add chatbot workaround for Meta Llama-3 support via stop token addition.
• Add logic to better handle model maximum context length errors with automated downsizing.
• Error handling and auto-retry for model changes on LLM.

v0.14.3 - Resize Control

• Add intuitive UI control at top of user input area to allow user to resize text input box.

v0.14.2 - Chatbot Stock RAG

• Add error checking and help for /stock {company} command.
• Allow user input textarea to be resized vertically.

v0.14.1 - Chatbot Baseprompt

• Fixed bug with baseprompt updates to respond to saved Settings or new sessions.
• Updated baseprompt to include date and guidance for complex and open-ended questions.
• Add TZ local timezone environmental variable to ensure correct date in baseprompt.

Full Changelog: v0.14.0...v0.14.1

v0.14.0 - Chatbot Controls

• Added ability to change LLM Temperature and MaxTokens in settings.
• Added optional prompt settings read-only options to allow viewing but prevent changes (PROMPT_RO=true).

Full Changelog: v0.13.0...v0.14.0

v0.13.0 - Use Weaviate for RAG

What's Changed

• Moved from Qdrant to Weaviate - This externalizes the sentence transformation work and lets the chatbot run as a smaller service. Activate by setting WEAVIATE_HOST to the address of the DB
• Added "References" text to output from /rag queries.
• Added ONESHOT environmental variable that if True will remove conversation threading allowing each query to be answered as a standalone sessions.
• Added RAG_ONLY environmental variable that if True will assume all queries should be directed to the default RAG database as set by WEAVIATE_LIBRARY.
• See #5

docker run \
    -d \
    -p 5000:5000 \
    -e PORT=5000 \
    -e OPENAI_API_BASE="http://localhost:8000/v1" \
    -e ONESHOT="true" \
    -e RAG_ONLY="false" \
    -e WEAVIATE_HOST="localhost" \
    -e WEAVIATE_LIBRARY="tinyllm" \
    -v $PWD/.tinyllm:/app/.tinyllm \
    --name chatbot \
    --restart unless-stopped \
    jasonacox/chatbot

Full Changelog: v0.12.5...v0.13.0

v0.12.5 - Chatbot LLM Model

• Added logic to poll LLM for model list. If only one model is available, use that. Otherwise verify the user requested model is available.
• Chatbot UI now shows model name and adds responsive elements to better display on mobile devices.
• Add encoding user prompts to correctly display html code in Chatbot.
• Fix chat.py CLI chatbot to handle user/assistant prompts for vLLM.

v0.12.4 - Chatbot Fixes

• Add encoding user prompts to correctly display html code in Chatbot.
• Fix chat.py CLI chatbot to handle user/assistant prompts for vLLM.

Full Changelog: v0.12.3...v0.12.4

v0.12.3 - Extract from URL

• Bug fix for handle_url_prompt() to extract text from URL.

Full Changelog: v0.12.2...v0.12.3

v0.12.2 - Misc Improvements

v0.12.2

• Speed up command functions using aiohttp.
• Fix prompt_expand for rag command.
• Added topic option to /news command.

v0.12.1 - Performance Improvements

• Speed up user prompt echo. Immediately send to chat windows instead of waiting for LLM stream to start.
• Optimize message handling dispatching using async.
• Use AsyncOpenAI for non-streamed queries.

Full Changelog: v0.12.0...v0.12.2

v0.12.0 - FastAPI and Uvicorn

Chatbot - v0.12.0 - FastAPI and Uvicorn

• Ported Chatbot to the async FastAPI and Uvicorn ASGI high speed web server implementation (#3).
• Added /stats page to display configuration settings and current stats (optional ?format=json)
• UI updated to help enforce focus on text entry box.
• Moved prompts.json and Sentence Transformer model location to a ./.tinyllm for Docker support.