Hebrew Grocery Item Classifier v2
Fine-tuned intfloat/multilingual-e5-small
for classifying Hebrew grocery shopping list items into 12 supermarket categories.
finetune code at https://github.com/davidit17/shopping_list_bot
What's new in v2
- Larger dataset — 1886 total items (1508 train / 378 test), up from ~600 in v1
- 4 training sources — manual lists, Excel grocery data, Gemini-labelled web data, and a ChatGPT-labelled set
- Synthetic augmentation — rule-based Hebrew item generator to balance under-represented categories
- Weak-class booster — extra hand-curated examples for משקאות, קפואים וסלטים, and רטבים וממרחים
- Per-class F1 tracking — evaluation now reports per-category F1 to surface weak spots
Categories
מוצרי חלב וביציםירקות ופירותלחם מאפים ודגניםחטיפים ומתוקיםניקיון טיפוח וחד פעמימשקאותמוצרים לאפייהבשר ודגיםקפואים וסלטיםיבשיםרטבים וממרחיםאחר
Usage
from transformers import pipeline
classifier = pipeline(
"text-classification",
model=f"davidit17/e5-grocery-finetuned-v2",
)
items = ["גבינה צהובה", "עגבניות שרי", "שמפו", "קוקה קולה", "חזה עוף"]
for item in items:
result = classifier(item)[0]
print(item, "→", result["label"], f"({result['score']:.2f})")
With confidence threshold (recommended)
CONFIDENCE_THRESHOLD = 0.5
def predict(text: str) -> str:
result = classifier(text)[0]
return result["label"] if result["score"] >= CONFIDENCE_THRESHOLD else "אחר"
Training
| Parameter |
Value |
| Base model |
intfloat/multilingual-e5-small |
| Max epochs |
50 (early stopping, patience=8) |
| Learning rate |
2e-5 |
| Weight decay |
0.01 |
| Warmup ratio |
0.1 |
| Batch size |
16 |
| Max sequence length |
32 |
| Train / test split |
80% / 20% stratified |
| Hardware |
GPU (CUDA) |
Data sources
| Source |
Description |
| Source A |
Manual Hebrew grocery lists from Israeli forum |
| Source B |
Excel-format grocery list, cleaned and melted |
| Source C |
Web grocery list labelled with Gemini |
| Source D |
ChatGPT-labelled items across all 12 categories |
| Source E |
Synthetic items generated with rule-based Hebrew augmentor |
| Source F |
Hand-curated booster set for weak categories |
Evaluation
Overall
| Metric |
Base (zero-shot) |
Fine-tuned v2 |
| Accuracy |
0.0556 |
0.6878 |
| Weighted F1 |
0.0146 |
0.6758 |
Per-class F1 (fine-tuned v2)
| Category |
F1 |
מוצרי חלב וביצים |
0.8333 |
ירקות ופירות |
0.5000 |
לחם מאפים ודגנים |
0.6897 |
חטיפים ומתוקים |
0.8308 |
ניקיון טיפוח וחד פעמי |
0.7292 |
משקאות |
0.8125 |
מוצרים לאפייה |
0.8000 |
בשר ודגים |
0.7317 |
קפואים וסלטים |
0.4091 |
יבשים |
0.3243 |
רטבים וממרחים |
0.1429 |
אחר |
0.8182 |
Limitations
- Primarily covers Israeli supermarket conventions and Hebrew product naming.
- Brand-specific or very niche items may fall back to
אחר.
- Low-confidence predictions (score < 0.5) should be treated as
אחר.
- Synthetic training examples may not fully reflect natural shopping list variation.
- The
אחר category is a catch-all and may have lower precision than domain-specific categories.
🚀 ONNX Support
This repository now includes optimized ONNX versions of the model for efficient inference.
📊 CPU Performance Comparison
The repository includes the original PyTorch model as well as optimized ONNX versions for faster CPU inference.
| Model |
Accuracy |
Weighted F1 |
Macro F1 |
Avg. Latency (ms) |
Throughput (samples/sec) |
| PyTorch (safetensors) |
0.6878 |
0.6758 |
0.6351 |
19.83 |
50.43 |
| ONNX (FP32) |
0.6878 |
0.6758 |
0.6351 |
9.80 |
102.09 |
| ONNX (INT8) |
0.6402 |
0.6184 |
0.5701 |
6.70 |
149.21 |
🚀 Improvements over the PyTorch model
ONNX (FP32)
- ⚡ 2.02× lower inference latency (19.83 ms → 9.80 ms)
- 🚀 2.02× higher throughput (50.4 → 102.1 samples/sec)
- ✅ No loss in accuracy or F1 score
ONNX (INT8)
- ⚡ 2.96× lower inference latency (19.83 ms → 6.70 ms)
- 🚀 2.96× higher throughput (50.4 → 149.2 samples/sec)
- 📉 Accuracy: −4.76 percentage points
- 📉 Weighted F1: −5.73 percentage points
Recommendation
- PyTorch (
safetensors) — Best for further fine-tuning and GPU inference.
- ONNX FP32 — Recommended for CPU deployment when maintaining the original model quality is important.
- ONNX INT8 — Best for latency-critical CPU applications where a small reduction in predictive performance is acceptable in exchange for nearly 3× faster inference.
Loading the ONNX model
from optimum.onnxruntime import ORTModelForFeatureExtraction
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(
"davidit17/e5-grocery-finetuned-v2"
)
model = ORTModelForFeatureExtraction.from_pretrained(
"davidit17/e5-grocery-finetuned-v2",
subfolder="onnx",
)
model = ORTModelForFeatureExtraction.from_pretrained(
"davidit17/e5-grocery-finetuned-v2",
subfolder="onnx-int8",
)
