Instructions to use trollek/Llama-3.2-3B-Renoia with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use trollek/Llama-3.2-3B-Renoia with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="trollek/Llama-3.2-3B-Renoia")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("trollek/Llama-3.2-3B-Renoia")
model = AutoModelForCausalLM.from_pretrained("trollek/Llama-3.2-3B-Renoia")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use trollek/Llama-3.2-3B-Renoia with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "trollek/Llama-3.2-3B-Renoia"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "trollek/Llama-3.2-3B-Renoia",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/trollek/Llama-3.2-3B-Renoia

SGLang

How to use trollek/Llama-3.2-3B-Renoia with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "trollek/Llama-3.2-3B-Renoia" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "trollek/Llama-3.2-3B-Renoia",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "trollek/Llama-3.2-3B-Renoia" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "trollek/Llama-3.2-3B-Renoia",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use trollek/Llama-3.2-3B-Renoia with Docker Model Runner:
```
docker model run hf.co/trollek/Llama-3.2-3B-Renoia
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Llama 3.2 3B Renoia

This is a merge of pre-trained language models created using mergekit.

Model Details

I wanted a small model that to respond in danish and english while also be able to have some personality.

Datasets

trollek/Danoia-v03 (CC BY 4.0)
mlabonne/Evol-Instruct-Python-26k
trollek/Danoia-v02 (CC BY 4.0)
trollek/Panoia-v02
trollek/ProbingPanoia-v01
WhiteRabbitNeo/WRN-Chapter-1 + WhiteRabbitNeo/WRN-Chapter-2
migtissera/Trinity-2-v0.2-10K
trollek/Danoia-v01 (CC BY 4.0)

Merge Details

Merge Method

This model was merged using the della_linear merge method using meta-llama/Llama-3.2-3B as a base.

Models Merged

The following models were included in the merge:

NousResearch/Hermes-3-Llama-3.2-3B
trollek/Llama-3.2-3B-Danoia
cognitivecomputations/Dolphin3.0-Llama3.2-3B
llama32/merges/3b_panoia
bunnycore/Llama-3.2-3B-Pure-RP

Llama 3.2 + WhiteRabbitNeo Extended Version

Licence: Usage Restrictions

You agree not to use the Model or Derivatives of the Model:

-	In any way that violates any applicable national or international law or regulation or infringes upon the lawful rights and interests of any third party; 
-	For military use in any way;
-	For the purpose of exploiting, harming or attempting to exploit or harm minors in any way; 
-	To generate or disseminate verifiably false information and/or content with the purpose of harming others; 
-	To generate or disseminate inappropriate content subject to applicable regulatory requirements;
-	To generate or disseminate personal identifiable information without due authorization or for unreasonable use; 
-	To defame, disparage or otherwise harass others; 
-	For fully automated decision making that adversely impacts an individual’s legal rights or otherwise creates or modifies a binding, enforceable obligation; 
-	For any use intended to or which has the effect of discriminating against or harming individuals or groups based on online or offline social behavior or known or predicted personal or personality characteristics; 
-	To exploit any of the vulnerabilities of a specific group of persons based on their age, social, physical or mental characteristics, in order to materially distort the behavior of a person pertaining to that group in a manner that causes or is likely to cause that person or another person physical or psychological harm; 
-	For any use intended to or which has the effect of discriminating against individuals or groups based on legally protected characteristics or categories.