Opensource LLM RAG Chatbot

• Project : An NLP project aimed at crafting a chatbot capable of answering questions sourced from provided documents. It leverages open-source large language models (LLM) and retrieval augmented generation (RAG) techniques for this purpose.
• Author: Kavindu Kalinga
• Tools: ollama , langchain ,ipynb
• Concepts used: Natural Language Processing(NLP), Large Language Models(LLM), Retrieval Augmented Generation (RAG)
• Models used: llama3:latest, nomic-embed-text:latest, mxbai-embed-large:latest
• Languages: python3
• Packages and APIs: crewai,faiss-cpu, langchain, chromadb, pypdf, pytest

Content

• Opensource LLM RAG Chatbot
  • Content
  • Ollama
  • Ollama models
    • Llama3
    • mxbai-embed-large
    • nomic-embed-text
  • HuggingFace
    • sentence-transformers/all-MiniLM-l6-v2
    • Salesforce/SFR-Embedding-Mistral
  • Contributors

Ollama

Download and Install Ollama : https://ollama.com/download

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Usage:
  ollama [flags]
  ollama [command]

# Available Commands:
  serve       Start ollama
  create      Create a model from a Modelfile
  show        Show information for a model
  run         Run a model
  pull        Pull a model from a registry
  push        Push a model to a registry
  list        List models
  cp          Copy a model
  rm          Remove a model
  help        Help about any command

# Flags:
  -h, --help      help for ollama
  -v, --version   Show version information

Ollama models

Check Ollama models here: https://ollama.com/library

ollama pull llama3:latest
ollama pull mxbai-embed-large
ollama pull nomic-embed-text:latest
ollama list

Make sure Ollama is running running on http://127.0.0.1:11434/.
Otherwise run:

ollama serve

#Example Case:
curl -X POST http://localhost:11434/api/generate -d '{
  "model": "llama3",
  "prompt":"Why is the sky blue?"
 }'

# Run On Terminal
ollama run llama3:latest
>>> Send a message (/? for help)
>>> /bye

Llama3

mxbai-embed-large

As of March 2024, this model archives SOTA performance for Bert-large sized models on the MTEB. It outperforms commercial models like OpenAIs text-embedding-3-large model and matches the performance of model 20x its size.

mxbai-embed-large was trained with no overlap of the MTEB data, which indicates that the model generalizes well across several domains, tasks and text length.

Source: https://ollama.com/library/mxbai-embed-large

nomic-embed-text

nomic-embed-text is a large context length text encoder that surpasses OpenAI text-embedding-ada-002 and text-embedding-3-small performance on short and long context tasks.

Source: https://ollama.com/library/mxbai-embed-large

HuggingFace

Compare HuggingFace models here : https://huggingface.co/spaces/mteb/leaderboard

sentence-transformers/all-MiniLM-l6-v2

Salesforce/SFR-Embedding-Mistral

The Best for Retrieval Average (15 datasets)

Contributors

Author : Kavindu Kalinga