-
-
Notifications
You must be signed in to change notification settings - Fork 82
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(docs): RFC on Searching for Focus Component on UI #469
base: main
Are you sure you want to change the base?
feat(docs): RFC on Searching for Focus Component on UI #469
Conversation
* Add 2 columns in screenshot table to store png_diff_data and png_mask_diff_data. * CRUD now supports calculation and save screenshots diff data on the flight.
* SAVE_SCREENSHOT_DIFF indicates that 2 neighbors screenshot will be compared and the difference will be saved to db
Thank you @LaPetiteSouris ! As per our conversation last week, this sounds directionally correct. Before implementing, can you please coordinate with @FFFiend to parallelize the work? 🙏 The first step of the problem formulation is:
@FFFiend can you please suggest some next steps once this has been implemented? |
Thanks With pleasure. This week I'll come up with a kind of skeleton code proposals to make sure the this is compatible with future fine-tuning, completion provider. If @FFFiend has time, please be one of the reviewers for that coming code/skeleton proposal for the next steps. @FFFiend how far are you off to finish FineTuning. Can I wrap up another round of review ? I would love to review CompletionProvider as well when it is available. |
#379 is ready for review. All done. Fine-tuning (#453 is too, however based on the discussion here and our steps moving forward, I think that PR is an example of a failure mode of fine-tuning, i.e showing the performance of the Davinci GPT model on the basic bare-bones iteration of Action, Window pairs) |
What kind of change does this PR introduce?
An RFC in attempt to solve #421 and to provide a direction for #442
Summary
When working on #421, I realize that there are a few missing pieces of the puzzle in an attempt to evaluate a prediction of the models. Mainly, we do not have yet the definition of "how good and useful" a prediction is. Without this, it is hard to evaluate and see if a model provides correct prediction or not.
Somehow, this problem is entangled with the fact that we need to come up with a prompt strategy and a strategy to summarize a
window
data into useful hints for the models.This PR provides a baseline for discussion on how we can decompose the windows into useful elements which later can serve not only model evaluation process, but also potentially prompt building as well as to some extend, RFLHF process.
Checklist
How can your code be run and tested?
The core RFC file should be reviewed/discussed. No code is written yet at the moment
Other information
openadapt.docs
.