Hew-Qwen3.5-4B — A Code Model for the Hew Programming Language

This is Qwen3.5-4B with a LoRA adapter merged in, fine-tuned to write valid Hew code.

Hew is a compiled, actor-based programming language for building resilient services. No LLM has seen Hew in pretraining — the language didn't exist when they were trained. This model was fine-tuned on ~1,900 compiler-validated samples to produce code that actually compiles.

80% compiler pass rate across 39 eval prompts spanning actors, supervisors, generators, state machines, wire types, pattern matching, concurrency, and more.

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model = AutoModelForCausalLM.from_pretrained(
    "sleppistan/hew-qwen3.5-4b",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("sleppistan/hew-qwen3.5-4b")

messages = [
    {"role": "system", "content": "You are an expert Hew programmer. Write complete, correct Hew source code."},
    {"role": "user", "content": "Write a Hew actor that accumulates integers and returns the running total on request."},
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=1024, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

Training Details

• Base model: Qwen/Qwen3.5-4B (4.2B parameters)
• Method: LoRA (rank 16, alpha 32, dropout 0.05) on all attention + MLP projections
• Training data: 1,926 samples in ChatML format, every code sample validated by hew check
• Epochs: 2
• Final loss: 0.163, token accuracy: 98.1%
• Hardware: AMD Radeon 780M (16 GB VRAM), ~4 hours per training run
• Precision: bfloat16 (no quantization during training)

Training History

Version	Samples	Config	Pass Rate
v7	1,827	1 epoch, r=16	53%
v8	1,866	1 epoch, r=16	69%
v9	1,898	1 epoch, r=16	68%
v11	1,926	2 epochs, r=16	80%
v12	1,926	1 epoch, r=32	71%

Key finding: 2 epochs on the same corpus beat more correction data or higher LoRA rank.

What This Model Can Do

• Write complete Hew programs with actors, supervisors, and structured concurrency
• Generate state machines, generators, wire types, and pattern matching
• Produce code that compiles 80% of the time against the Hew compiler

What It Gets Wrong (the 20%)

• Type errors from Rust-like APIs that don't exist in Hew (.iter(), .contains(), Vec::new())
• Move semantics errors with actor message sends
• Occasional prose output instead of raw code

Related Resources

• LoRA adapter (GGUF): sleppistan/hew-lora-qwen3.5-4b — for use with llama.cpp
• Blog post: Teaching an LLM a Language It Has Never Seen
• Hew language: hew-lang.org