missing biases?
#1
by
ztsvvstz - opened
Hi, after some patching vllm I got it running on two rtx3090
However, VLLM was complaining about missing keys like "w2_bias"
Just for testing I made it skip those checks and the model suprisingly starts but produces just gibberish.
I wonder did you forget to include those keys or any other directional hint on how to properly run this via vllm? :)