Adding support for quantization and fast inference server

[rostandk]

DeepSeek-VL 7B seems to be the awesome vision model that nobody is talking about. It is better than Llava 1.6 34B in VQA and OCR. It consistently return JSON when asked and just seems to have a great textual capability. Any recommendation on how to wrap DeepSeek-VL in a fast inference server and take for spin at scale ?

There are some issues opened here:
InternLM/lmdeploy#1321
sgl-project/sglang#297