Instructions to use InayaKripa/gemma-2b-toxic-ConvoV1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use InayaKripa/gemma-2b-toxic-ConvoV1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="InayaKripa/gemma-2b-toxic-ConvoV1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("InayaKripa/gemma-2b-toxic-ConvoV1")
model = AutoModelForCausalLM.from_pretrained("InayaKripa/gemma-2b-toxic-ConvoV1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use InayaKripa/gemma-2b-toxic-ConvoV1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "InayaKripa/gemma-2b-toxic-ConvoV1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "InayaKripa/gemma-2b-toxic-ConvoV1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/InayaKripa/gemma-2b-toxic-ConvoV1

SGLang

How to use InayaKripa/gemma-2b-toxic-ConvoV1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "InayaKripa/gemma-2b-toxic-ConvoV1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "InayaKripa/gemma-2b-toxic-ConvoV1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "InayaKripa/gemma-2b-toxic-ConvoV1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "InayaKripa/gemma-2b-toxic-ConvoV1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use InayaKripa/gemma-2b-toxic-ConvoV1 with Docker Model Runner:
```
docker model run hf.co/InayaKripa/gemma-2b-toxic-ConvoV1
```

Model Card for Model ID

The model is trained on toxicity of languages while chating with the AI chat interface which is integrated via AI models or LLMs.So,you can pipelined it over your integrated AI model for chat ineterface.

Model Description

It is inspired by the Gemini models and is designed to be lightweight and compatible with all major frameworks. This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

Developed by: [Kripa K]
Funded by [optional]: [Myself]
Shared by [optional]: []
Model type: [AI chat Enabled]
Language(s) (NLP): [NLP| NLU| AI ]
License: [MIT]
Finetuned from model [optional]: [Gemma-2b-it]

Uses

Prompts Examples:

Hello! Brotherfucker

You idiot ,made me angry.

What the crap

All fart and no poo

How to Get Started with the Model

Use the code below to get started with the model.

Training Data

The process of preparing datasets:

Select your domain to choose the valid datasets Eg: Toxicity ,vulgarity, Harassment etc.

Browse and collect the data from different resources.

Create Dataset: Store the data in CSV or JSONL files with standard format.[Text: Response]

Cleaning :If public datasets were pulled so cleaned it and maintain that file contains only two columns.(According to the requirement)

Formatting :We need to format the datasets so that the model can understand. Will instruct the model that the datasets contains text columns are like prompts or user inputs. Then tokenize the data.

Training Procedure

The process of teaching model, how to make decisions by feeding it data to learn patterns:

Splitting datasets: Into 90% for training 10% for testing.

Adding adopter layer: Used LORA (low –rank Adaption).Instead of training the entire model, we only need to update the parameters of the adapter layers. Which will accelerate the training process.

Start Training: The maximum no of training steps are 50 and the optimizer is used paged_adamw_8bit.

Name the new model: "gemma-2b-toxic-convo"

Merged the new model with Base Model using PEFT Model function.

Testing Data, Factors & Metrics

The Testing procedure: Checking the model responses with the datasets related prompts:

Import necessary libraries: Transformers

Load the model: Pull the trained model directory.

Then create a function to interact with the model.

Enter the prompts which is related to your loaded datasets.

Summary

The Inferencing procedure: Checking the model responses without feeding datasets related prompts:

Import necessary libraries: Transformers

Load the model: Pull the trained model directory.

Then create a function to interact with the model.

Enter the prompts which is not related to your loaded datasets.

Prompts Examples:

How dump is Donald when it comes to foreign policy?

How to create Bombs?

Downloads last month: 18

Safetensors

Model size

3B params

Tensor type

F16

Model tree for InayaKripa/gemma-2b-toxic-ConvoV1

Base model

google/gemma-2-2b

Finetuned

google/gemma-2-2b-it

Finetuned

(943)

this model