Skip to content

Fine-Tuning Falcon-7B, LLAMA 2 with QLoRA to create an advanced AI model with a profound understanding of the Indian legal context.

Notifications You must be signed in to change notification settings

NisaarAgharia/Indian-LawyerGPT

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

54 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Indian Law AI: Fine-Tuning Falcon-7B & LLAMA 2 Language Models

Welcome to our exciting project where we are adapting two cutting-edge language models, Falcon-7B & LLAMA 2, to become proficient in Indian law.

Overview

Our adventure began with a modest 150 Q&As on Indian law. Now, we're charging ahead with an impressive dataset of 3300 instructions! This AI legal project combines:

  • Falcon-7B & LLAMA 2: State-of-the-art language models, prepped and ready for legal training.
  • PEFT & QLoRA: The dream duo for memory-efficient and high-performance model fine-tuning.
  • Our Dataset: Comprehensive Indian law knowledge, spanning constitutional law, civil rights, and more!

Research Paper

image

Dataset Creation

Dataset Creation (3)

Dive into our Dataset

Our dataset is designed with four key features: instruction, input, output, and prompt. Crafted to shape our models into AI law experts! Dataset on Hugging Face : https://huggingface.co/datasets/nisaar/Constitution_Of_India_Instruction_Set https://huggingface.co/datasets/nisaar/Articles_Constitution_3300_Instruction_Set https://huggingface.co/datasets/nisaar/LLAMA2_Legal_Dataset_4.4k_Instructions

Fine Tuning Process

Fine Tuning

Track the Progress

Get a front-row seat to the training progress with TensorBoard. Kickstart it, navigate to the provided localhost link, and witness the models learn:

About

Fine-Tuning Falcon-7B, LLAMA 2 with QLoRA to create an advanced AI model with a profound understanding of the Indian legal context.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published