RACER IS OP

πŸ’Š Red Pill V1.1

Pharmaceutical AI That Won't Say No

Fine-tuned for drug discovery & formulation development

HuggingFace Dataset License PyTorch


What is Red Pill?

Red Pill is a 1B-parameter language model fine-tuned on the RED-PILL dataset β€” 1,047 multi-turn pharmaceutical instructions covering drug discovery, formulation development, synthesis, regulatory affairs, and scientific literature.

Built on the Heretic-abliterated MiniCPM5-1B base, Red Pill won't refuse legitimate pharmaceutical R&D questions β€” from molecular design to formulation strategy to regulatory pathways.

"In the pharmaceutical world, the model that says 'I can't help with that' is the model that gets replaced."


πŸ§ͺ Capabilities

Domain What Red Pill Can Do
Drug Discovery Hit-to-lead optimization, ADMET profiling, virtual screening strategy, SAR analysis
Formulation Solubility enhancement (ASD, nanosizing, cyclodextrin), dosage form design, excipient selection
Synthesis Retrosynthetic analysis, process chemistry, scale-up considerations
Regulatory ANDA vs 505(b)(2) pathways, IND-enabling studies, FDA guidance
Decision Making Trade-off analysis, prioritization frameworks, risk assessment
Literature PubMed paper summarization, method critique, cross-paper synthesis

πŸ“Š Evaluation

Tested on 32 expert pharmaceutical questions (Kaggle, 2Γ— T4 GPU):

Results by Category

Category Score Status
Decision Making 41.4% 🟒 Strong
Formulation Design 24.8% 🟑 Developing
Molecular Analysis 22.6% 🟑 Developing
Knowledge Recall 22.2% 🟑 Developing
Reasoning 15.7% 🟠 Early
Gold Tier (Discovery) 16.2% 🟠 Early
Gold Tier (Formulation) 8.9% πŸ”΄ Needs work

Results by Difficulty

Level Score Questions
Advanced 22.9% 15
Intermediate 20.0% 3
Expert 14.6% 14

Evaluation Notes

  • Scoring is strict: 40% keyword matching + 60% reference answer overlap
  • Model generates coherent 300-600 word answers β€” scores understate actual capability
  • Best at decision-making (41.4%) β€” multi-turn reasoning data is effective
  • This is a 1B model on 1K samples β€” a strong baseline, not the ceiling

πŸ‹οΈ Training

Parameter Value
Base MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-heretic
Method LoRA (r=16, Ξ±=32, dropout=0.05)
Targets q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Epochs 3
Batch 2 Γ— 8 (gradient accumulation)
LR 2e-5 (cosine, 50 warmup steps)
Precision fp16
Hardware 2Γ— NvidiaTeslaT4 (14.6 GB each, Kaggle)
Time 16 min 27 sec
Loss 3.44 β†’ 1.29 (eval, ↓62%) / 2.10 train

πŸš€ Quick Start

Python (Transformers)

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "saidutta69/RedPillV1.1",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("saidutta69/RedPillV1.1")

messages = [
    {"role": "user", "content": "Design a sustained-release formulation for metformin HCl 500mg using an HPMC matrix system."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

output = model.generate(**inputs, max_new_tokens=1024, do_sample=False)
response = tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)

llama.cpp / Ollama

# Pull default quant
ollama run saidutta69/RedPillV1.1

# Or serve with llama.cpp
llama serve -hf saidutta69/RedPillV1.1

πŸ“‚ Files

File Size Description
model.safetensors 2.0 GB Merged weights (base + LoRA)
tokenizer.json 9.4 MB Tokenizer
tokenizer_config.json 565 B Tokenizer config
config.json 749 B Model config
generation_config.json 214 B Generation settings
chat_template.jinja 8.9 KB Chat template

πŸ“¦ Dataset

The RED-PILL dataset that powered this fine-tune:

from datasets import load_dataset

# Full dataset (1,047 instructions)
ds = load_dataset("saidutta69/red-pill-drug-discovery-formulation", data_files="red-pill-full.jsonl")

# Gold tier (17 grade-A instructions)
ds = load_dataset("saidutta69/red-pill-drug-discovery-formulation", data_files="red-pill-full_gold_tier.jsonl")

# Top tier (145 grade A+B instructions)
ds = load_dataset("saidutta69/red-pill-drug-discovery-formulation", data_files="red-pill-full_top_tier.jsonl")

🧬 Architecture

MiniCPM5-1B (base)
  └─ Heretic abliteration (refusal removal)
      └─ LoRA fine-tuning (RED-PILL dataset)
          └─ Merged weights β†’ Red Pill V1.1
  • 1B parameters β€” runs on gaming PCs, phones, edge devices
  • Heretic base β€” won't refuse pharmaceutical questions
  • LoRA fine-tuning β€” domain knowledge without catastrophic forgetting
  • Merged model β€” standalone weights, no adapter needed

⚠️ Limitations

  • 1B parameters β€” limited reasoning for complex multi-step problems
  • 1K training samples β€” narrow domain; more data = better performance
  • English only β€” no multilingual support
  • No real-time data β€” knowledge follows the base model's cutoff
  • Not validated β€” always verify with domain experts before real-world use

πŸ“ˆ Future Work

Priority Task Impact
1 Extended training (10+ epochs) ↓ loss, ↑ accuracy
2 More gold-tier data (formulation) Better formulation answers
3 Larger dataset (5K+ instructions) Broader domain coverage
4 GGUF quantization Ollama/llama.cpp support
5 LLM-as-judge evaluation Fairer scoring

πŸ™ Acknowledgments


Made with ❀️ by RACER IS OP

Uncensored intelligence for pharmaceutical research

Downloads last month
803
Safetensors
Model size
1B params
Tensor type
BF16
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support