FineTuning

[FFFiend]

What kind of change does this PR introduce?
addresses #69

Summary

Checklist

• My code follows the style guidelines of OpenAdapt
• I have performed a self-review of my code
• If applicable, I have added tests to prove my fix is functional/effective
• I have linted my code locally prior to submission
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation (e.g. README.md, requirements.txt)
• New and existing unit tests pass locally with my changes

How can your code be run and tested?

Other information

[FFFiend]

Refactoring and OpenAI Tuner completed, along with some additional changes to function names.

[FFFiend]

TODO: add metrics infrastructure for finetuning

Some useful repos: https://github.com/lxe/simple-llm-finetuner, and https://github.com/kuleshov-group/llmtune/tree/main

lora.py This links to theirlora.py file, which I believe we intended to at least investigate in the past. (#208)
The docstring of the class states that

class QuantLoraModel(torch.nn.Module):
    """
    Creates Low Rank Adapter (Lora) model from a pretrained transformers model.

    Args:
        model ([`transformers.PreTrainedModel`]): The model to be adapted.
        config ([`LoraConfig`]): The configuration of the Lora model.

    Returns:
        `torch.nn.Module`: The Lora model.
.
.
.

so this might be a direct solution to that issue too.

As the name of the repo suggests, it's meant for consumer grade GPU finetuning, we can surely implement something for CPU too.

[LaPetiteSouris]

Just thinking out loud: should this file be in /tmp instead of current dir ? Do we actually need this file permanently ?

[FFFiend]

We could just pass in a shell command to delete the file that was generated using subprocess immediately after the file has been uploaded to OpenAI for finetunign. WDYT?

[LaPetiteSouris]

Neat pick: May be it is a good idea to put a small README.md file with a few lines of example for documentation.

From a new comer point of view, it is hard to know which method should be called first and which one later.

[FFFiend]

Noted, will add this asap.

[LaPetiteSouris]

Thanks for implementing this. This looks great ! 👍 💯

[FFFiend]

TODO: add tests, address comments.

[FFFiend]

Model used to finetune: Davinci.
Current output on a snippet of the same recording that I finetuned the model on:

"({'name': 'move', 'mouse_x': 359.66015625, 'mouse_y': 118.9765625, 'element_state': {}}, {'title': 'Terminal openadapt \u2014 poetry shell \u25b8 Python \u2014 124\u00d740', 'left': 283, 'top': 109, 'width': 878, 'height': 595, 'window_id': 1129})

({'name': 'move', 'mouse_x': 363.30859375, 'mouse_y': 107.890625, 'element_state': {}}, {'title': 'Terminal openadapt \u2014 poetry shell \u25b8 Python \u2014 124\u00d740', 'left': 283, 'top': 109, 'width': 878, 'height': 595, 'window_id': 1129})

({'name': 'move', 'mouse_x': 368.55078125, 'mouse_y': 99.62890625, 'element_state': {}}, {'title': 'Terminal openadapt \u2014 poetry shell \u25b8 Python \u2014 124\u00d740', 'left': 283, 'top': 109, 'width': 878, 'height': 595, 'window_id': 1129})

({'name': 'move', 'mouse_x': 373.83203125, 'mouse_y': 90.75390625, 'element_state': {}}, {'title': 'Terminal openadapt \u2014 poetry shell \u25b8 Python \u2014 124\u00d740', 'left': 283, 'top': 109, 'width': 878, 'height': 595, 'window_id': 1129})

({'name': 'move', 'mouse_x': 378.2265625, 'mouse_y': 82.43359375, 'element_state': {}}, {"

expected output:

({'name': 'move', 'mouse_x': 356.796875, 'mouse_y': 124.1640625, 'element_state': {}}, {'title': 'Terminal openadapt \u2014 poetry shell \u25b8 Python \u2014 124\u00d740', 'left': 283, 'top': 109, 'width': 878, 'height': 595, 'window_id': 1129})

({'name': 'move', 'mouse_x': 361.0234375, 'mouse_y': 112.890625, 'element_state': {}}, {'title': 'Terminal openadapt \u2014 poetry shell \u25b8 Python \u2014 124\u00d740', 'left': 283, 'top': 109, 'width': 878, 'height': 595, 'window_id': 1129})

({'name': 'move', 'mouse_x': 362.83203125, 'mouse_y': 109.265625, 'element_state': {}}, {'title': 'Terminal openadapt \u2014 poetry shell \u25b8 Python \u2014 124\u00d740', 'left': 283, 'top': 109, 'width': 878, 'height': 595, 'window_id': 1129})

({'name': 'move', 'mouse_x': 364.9765625, 'mouse_y': 103.36328125, 'element_state': {}}, {'title': 'Terminal openadapt \u2014 poetry shell \u25b8 Python \u2014 124\u00d740', 'left': 283, 'top': 109, 'width': 878, 'height': 595, 'window_id': 1129})

({'name': 'move', 'mouse_x': 366.78515625, 'mouse_y': 99.73828125, 'element_state': {}}, {'title': 'Terminal openadapt \u2014 poetry shell \u25b8 Python \u2014 124\u00d740', 'left': 283, 'top': 109, 'width': 878, 'height': 595, 'window_id': 1129})

UPDATE: Need to try fine-tuning the newly available gpt-3.5-turbo-0613 because you can use generate ChatCompletions on it.

[abrichr]

Can you please move this to tests/assets/fixtures.json or similar?

[FFFiend]

TODO: flake8 errors before merging