Instructions to use TheDrummer/Valkyrie-49B-v1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use TheDrummer/Valkyrie-49B-v1-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf TheDrummer/Valkyrie-49B-v1-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf TheDrummer/Valkyrie-49B-v1-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf TheDrummer/Valkyrie-49B-v1-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf TheDrummer/Valkyrie-49B-v1-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf TheDrummer/Valkyrie-49B-v1-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf TheDrummer/Valkyrie-49B-v1-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf TheDrummer/Valkyrie-49B-v1-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf TheDrummer/Valkyrie-49B-v1-GGUF:Q4_K_M
Use Docker
docker model run hf.co/TheDrummer/Valkyrie-49B-v1-GGUF:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use TheDrummer/Valkyrie-49B-v1-GGUF with Ollama:
ollama run hf.co/TheDrummer/Valkyrie-49B-v1-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use TheDrummer/Valkyrie-49B-v1-GGUF with Docker Model Runner:
docker model run hf.co/TheDrummer/Valkyrie-49B-v1-GGUF:Q4_K_M
- Lemonade
How to use TheDrummer/Valkyrie-49B-v1-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull TheDrummer/Valkyrie-49B-v1-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Valkyrie-49B-v1-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Join our Discord! https://discord.gg/Nbv9pQ88Xb
More than 5500 members of helpful, LLM enthusiasts! A hub for players and makers alike!
We need testers!
Live in OpenRouter! (Powered by Parasail.io)
Drummer proudly presents...
Valkyrie 49B v1
Description
it swears unprompted 10/10 model
... characters work well, groups work well, scenarios also work really well so great model overall
This is pretty exciting though. GLM-4 already had me on the verge of deleting all of my other 32b and lower models. I got to test this more but I think this model at Q3m is the death blow lol
Smart Nemotron 49b learned how to roleplay
Even without thinking it rock solid at 4qm.
Without thinking is like 40-70b level. With thinking is 100+b level
This model would have been AGI if it were named properly with a name like "Bob". Alas, it was not.
I think this model is nice. It follows prompts very well. I didn't really note any major issues or repetition
Yeah this is good. I think its clearly smart enough, close to the other L3.3 70b models. It follows directions and formatting very well. I asked it to create the intro message, my first response was formatted differently, and it immediately followed my format on the second message. I also have max tokens at 2k cause I like the model to finish it's thought. But I started trimming the models responses when I felt the last bit was unnecessary and it started replying closer to that length. It's pretty much uncensored.
Usage
- Llama 3 Chat Template
- <think> capable upon prefill or
detailed thinking onon top of the system prompt
Links
- Original: https://huggingface.co/TheDrummer/Valkyrie-49B-v1
- GGUF: https://huggingface.co/TheDrummer/Valkyrie-49B-v1-GGUF
- iMatrix (recommended): https://huggingface.co/bartowski/TheDrummer_Valkyrie-49B-v1-GGUF
- EXL3: https://huggingface.co/collections/ReadyArt/thedrummer-valkyrie-49b-v1-exl3-682b634353aba9f40d906185
Special Thanks
- Thank you to the testers at BeaverAI! You da MVP!
- Thank you to each and everyone who donated and subscribed in Patreon and Ko-Fi to make our venture a little bit easier.
- Subscribe to my Patreon!
config-v1a
- Downloads last month
- 177
2-bit
3-bit
4-bit
5-bit
6-bit
8-bit
Model tree for TheDrummer/Valkyrie-49B-v1-GGUF
Base model
nvidia/Llama-3_3-Nemotron-Super-49B-v1