Any recipe for vLLM?

by hdnh2006 - opened 5 days ago

Hello!

Thanks for this fantastic release, any idea how to deploy it in 24GB of VRAM using vLLM omni? is this even possible?

Thanks in advance.

ModelsLab org 5 days ago

vLLM omni won't support this format, use custom code stack, we have in readme, it will work. or better ask claude to make it work, it will work.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment