Fine Tuning on Custom Data ipynb

[samarthsarin]

Can you please provide a ipynb notebook which shows steps for fine tuning this model on custom data?

[daleevans]

The entire process is just:

make your data, in the jsonl format like you get when you download the standard data
edit configs/train/finetune_lora.yaml to point to your new data file and set up your wandb/hf account info
possibly edit configs/deepspeed/ds_config.json depending on your local GPU/CPU/memory (batch sizes, and maybe set stage3_gather_16bit_weights_on_model_save and cpu offload)
run train.py

If you don't have wandb or hf, you may need to comment some lines out in train.py

[samarthsarin]

I got your point. Still for better documentation I would be really grateful if some jupyter nptebook can be provided as majority of the audience here is looking for one such fine tuning code. We all would be really grateful if you can provide one such code for fine tuning gpt4all in a jupyter notebook.

Thank you

[magedhelmy1]

Hi @zanussbaum, any advise on how to move forward with this?

[Dineshk011287]

I got your point. Still for better documentation I would be really grateful if some jupyter nptebook can be provided as majority of the audience here is looking for one such fine tuning code. We all would be really grateful if you can provide one such code for fine tuning gpt4all in a jupyter notebook.

+1
I am also looking for this. If any documentation or jupyter notebook would definitely help

[cebtenzzre]

Closing this issue as stale. A lot has changed since Nomic last trained a text completion model.