Instructions to use Nekochu/Luminia-13B-v3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries
PEFT
How to use Nekochu/Luminia-13B-v3 with PEFT:
```
Task type is invalid.
```

How to use Nekochu/Luminia-13B-v3 with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Nekochu/Luminia-13B-v3",
	filename="Luminia-13B-v3-IQ4_NL.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Inference
Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use Nekochu/Luminia-13B-v3 with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf Nekochu/Luminia-13B-v3:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf Nekochu/Luminia-13B-v3:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf Nekochu/Luminia-13B-v3:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf Nekochu/Luminia-13B-v3:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Nekochu/Luminia-13B-v3:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Nekochu/Luminia-13B-v3:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Nekochu/Luminia-13B-v3:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Nekochu/Luminia-13B-v3:Q4_K_M

Use Docker

docker model run hf.co/Nekochu/Luminia-13B-v3:Q4_K_M

LM Studio
Jan

vLLM

How to use Nekochu/Luminia-13B-v3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Nekochu/Luminia-13B-v3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nekochu/Luminia-13B-v3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Nekochu/Luminia-13B-v3:Q4_K_M

Ollama
How to use Nekochu/Luminia-13B-v3 with Ollama:
```
ollama run hf.co/Nekochu/Luminia-13B-v3:Q4_K_M
```

Unsloth Studio new

How to use Nekochu/Luminia-13B-v3 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Nekochu/Luminia-13B-v3 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Nekochu/Luminia-13B-v3 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Nekochu/Luminia-13B-v3 to start chatting

Docker Model Runner
How to use Nekochu/Luminia-13B-v3 with Docker Model Runner:
```
docker model run hf.co/Nekochu/Luminia-13B-v3:Q4_K_M
```

Lemonade

How to use Nekochu/Luminia-13B-v3 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Nekochu/Luminia-13B-v3:Q4_K_M

Run and chat with the model

lemonade run user.Luminia-13B-v3-Q4_K_M

List all available models

lemonade list

Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

DALL-E 3 prompt: a single seed growing slowly in laboratory in a desert sand, the single little plant try fight to reach light sun, while a little cute kitty feel the plant, cute 8k anime, digitral art, close up

Luminia v3 is good at reasoning to enhance Stable Diffusion prompt from short summary description, may output NSFW content.

LoRa is include and Quants: exllamav2 2.4bpw-h6, 4.25bpw-h6, 8.0bpw-h8 | GGUF Q4_K_M, IQ4_NL |

Prompt template: Alpaca

Output example tested In text-generation-webui

Input	base llama-2-chat	QLoRa
[question]: Create stable diffusion metadata based on the given english description. Luminia \n### Input:\n favorites and popular SFW	Answer: Luminia, a mystical world of wonder and magic 🧝‍♀️✨ A place where technology and nature seamlessly blend together ...	Answer! < lora:Luminari-10:0.8> Luminari, 1girl, solo, blonde hair, long hair, blue eyes, (black dress), looking at viewer, night sky, starry sky, constellation, smile, upper body, outdoors, forest, moon, tree, mountain, light particle ....

Output prompt from QLoRa to A1111/SD-WebUI:

Full Prompt

Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
Create stable diffusion metadata based on the given english description. Luminia

### Input:
favorites and popular SFW

### Response:

"Luminia" can be any short description, more info on my SD dataset here.

Training Details

Click to see details

Model Description

Train by: Nekochu, Model type: Llama, Finetuned from model Llama-2-13b-chat
Continue from the base of LoRA Luminia-13B-v2-QLora

Know issue: [issue]

Trainer

hiyouga/LLaMA-Efficient-Tuning

Hardware: QLoRA training OS Windows, Python 3.10.8, CUDA 12.1 on 24GB VRAM.

Training hyperparameters

The following hyperparameters were used during training:

num_epochs: 1.0
finetuning_type: lora
quantization_bit: 4
stage: sft
learning_rate: 5e-05
cutoff_len: 4096
num_train_epochs: 3.0
max_samples: 100000
warmup_steps: 0
train_batch_size: 1
distributed_type: single-GPU
num_devices: 1
warmup_steps: 0
rope_scaling: linear
lora_rank: 32
lora_target: all
lora_dropout: 0.15
bnb_4bit_compute_dtype: bfloat16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine

training_loss:

Framework versions

PEFT 0.9.0
Transformers 4.38.1
Pytorch 2.1.2+cu121
Datasets 2.14.5
Tokenizers 0.15.0

Downloads last month: 96

Safetensors

Model size

13B params

Tensor type

F16

Model tree for Nekochu/Luminia-13B-v3

Base model

meta-llama/Llama-2-13b-chat-hf

Adapter

(146)

this model

Adapters

2 models

Quantizations

1 model

Nekochu
/

Luminia-13B-v3