That's so cool!
I'm still working on getting it running in VLLM but after literally like 14 hours of battling it I got it to complete quantization. Once I figure that out I'll do the big one too.
· Sign up or log in to comment
