W4AFP8 for 5.3
#7
by jlondon - opened
Howdy. Do you plan on doing this same work for 5.3 once that is released? Your weights have been the only so far I've been able to successfully run on 8x H200 which is pretty awesome. Thanks!
Yes, I plan to do so. And we may also build a vllm compatible version.