Model params for GPU setup

[vovsgit]

Hello @nexus! How do I choose the parameters for the models for my GPU setup? I'm manipulate different params, as example n_ctx or n_batch, but is so long, and not effectivity. Maybe do you know formula or method for calculate optimal params for GPU.

I have 2 1080Ti with 11 gb memory, please give me advice ))

[snexus]

Hi,

Sorry, I am not aware of a formula that allows calculating it easily. Would start with monitoring the GPU usage (on Linux, you can use nvidia-smi tool) and optimise the parameters one by one.