Building Docker with model built in

[KDercksen]

Hi there,

The current version of the download_model.py script does not work due to the empty TENSORIZE_MODEL env check on line 50.

Once that is fixed, the weight_utils file in the vllm-base image does not exist - it seems there is some version mismatch going on with the vllm submodule and the new 1.0.0preview image.

Could you take a look?

[alpayariyak]

Thanks a lot for pointing this out! Should work now

[nuckcrews]

Looks like prepare_hf_model_weights is no longer available in the vllm submodule as well (see: weight_utils.py).

Im getting the following error after pulling the latest changes.

2.686 Traceback (most recent call last):                                                                               
2.686   File "/download_model.py", line 6, in <module>
2.686     from vllm.model_executor.model_loader.weight_utils import prepare_hf_model_weights, Disabledtqdm
2.686 ImportError: cannot import name 'prepare_hf_model_weights' from 'vllm.model_executor.model_loader.weight_utils' (/usr/local/lib/python3.10/dist-packages/vllm/model_executor/model_loader/weight_utils.py)

[alpayariyak]

Thanks @nuckcrews, will fix this shortly, sorry for the oversight!

[nuckcrews]

@alpayariyak any updates on this issue? I'm still seeing the same error.