openbmb

/

MiniCPM-V-4_5-AWQ

like 10

Follow

OpenBMB 2.32k

Hello,

I ran this model in vllm with command

 python3 -m vllm.entrypoints.openai.api_server
      --model openbmb/MiniCPM-V-4_5-AWQ
      --limit-mm-per-prompt.image 1
      --limit-mm-per-prompt.video 1
      --limit-mm-per-prompt.audio 1
      --gpu-memory-utilization 0.85
      --max-model-len 2K
      --max-num-seqs 10
      --max-num-batched-tokens 2K
      --host 0.0.0.0
      --port 8000
      --dtype float16
      --trust-remote-code

In Open WebUI, I tried with prompt tell me names of islands in indonesia. I am expecting name of several islands in Indonesia, but I got random response like this

<reserved_12>
I will call the 'list_islands Indonesia' function to get the names of islands in Indonesia.
<reserved_13>
<reserved_14>
python
list_islands(country='Indonesia')

<reserved_15>

And sometimes like this (I am expecting this without those reserved texts)

<reserved_12>
I will list some well-known islands in Indonesia for the user.
<reserved_13>
Sure, here are some popular islands in Indonesia:
1. Bali
2. Lombok
3. Sumbawa
4. Flores
5. Komodo
6. Sulawesi
7. Maluku
8. Papua
9. Sumatra
10. Borneo
11. Java
12. Nusa Tenggara

Any advice please