This project aims to convert clibrain/Llama-2-7b-ft-instruct-es model into a compatible llama.cpp GGUF model.
Content
-
Clone llama.cpp
git clone https://github.com/ggerganov/llama.cpp.git
-
Build llama.cpp
cd llama.cpp && \ make
-
Download test model
bash download_test_model.sh
-
Test build
llama.cpp/main -ins \ -f llama.cpp/prompts/alpaca.txt \ -t 8 \ -ngl 1 \ -m models/test/llama-2-7b.Q4_K_M.gguf \ -c 2048 \ -s 42 \ -n -1 \ --temp 0.7 \ --repeat-penalty 1.1 \ --color
-
Write a sample instruction:
write a haiku about programming
![Screenshot 2023-10-16 at 10 52 32 PM](https://private-user-images.githubusercontent.com/74464814/275706525-6dcd1c1b-c71e-4e28-ac54-23c20c61abac.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTgwNTEzOTMsIm5iZiI6MTcxODA1MTA5MywicGF0aCI6Ii83NDQ2NDgxNC8yNzU3MDY1MjUtNmRjZDFjMWItYzcxZS00ZTI4LWFjNTQtMjNjMjBjNjFhYmFjLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjEwVDIwMjQ1M1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWVhMThmZmUyNzM3YjY1MmM2MzM1ZWQ1OGM3ZGUzNGVhM2E4MGEzYzRiMzE4YjI2NmNlYWRkZjZjNjE5MWQyMWImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.oUVvAoAj8F_oBNU3xZzHnAfYVKHAiuf3_X-g-Fe8o8U)
To download the model clibrain/Llama-2-7b-ft-instruct-es
, run:
python scripts/download_hf_model.py clibrain/Llama-2-7b-ft-instruct-es
It should take around 20min to download (based on your internet speed)
![image](https://private-user-images.githubusercontent.com/74464814/275720698-3ee3846f-f2b3-4f73-b99e-5716658bcedc.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTgwNTEzOTMsIm5iZiI6MTcxODA1MTA5MywicGF0aCI6Ii83NDQ2NDgxNC8yNzU3MjA2OTgtM2VlMzg0NmYtZjJiMy00ZjczLWI5OWUtNTcxNjY1OGJjZWRjLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjEwVDIwMjQ1M1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWI3ZGI3YjQ1YTdhNmMyYWMzMmYyODQ3M2FkMTIzMmM3YTExNjE0OGNlM2IyZTcxODkzNDExZjE3ODg2ODkyNjgmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.WMei01X-NZvmVwFgcHqdHrAYe_dHgjs_Fg5QkGXAtoA)
After, you can check files with:
ls -lhia models/Llama-2-7b-ft-instruct-es/
![image](https://private-user-images.githubusercontent.com/74464814/275720897-13599a15-e502-47c7-983e-dcba847e733a.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTgwNTEzOTMsIm5iZiI6MTcxODA1MTA5MywicGF0aCI6Ii83NDQ2NDgxNC8yNzU3MjA4OTctMTM1OTlhMTUtZTUwMi00N2M3LTk4M2UtZGNiYTg0N2U3MzNhLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjEwVDIwMjQ1M1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPThkNTkyMDllZTM1MmZhODQ4OGQ5ZWZjOWI5ZDY0ZTg4NDM4MjYyYzhmMTViY2RmMmIwZWQ3Nzg0ZjJkN2RjOWUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.cp67NFFrsL7z12TsoIeDvZXIVWTNMbuJPeOmY2zLbyQ)
Pre-commit is already part of this project dependencies. If you would like to installed it as standalone run:
pip install pre-commit
To activate pre-commit run the following commands:
- Install Git hooks:
pre-commit install
- Update current hooks:
pre-commit autoupdate
To test your installation of pre-commit run:
pre-commit run --all-files