my420254/hipro-lora-tweeteval

HiPro-LoRA — TWEETEVAL

Adaptive Hierarchical State Pooling + Tail-aware Prototype Memory Bank for low-resource sentiment classification.

Model Details

Item	Value
Base model	roberta-base
LoRA rank	16
LoRA alpha	32
Num labels	3
Labels	negative, neutral, positive
Test Macro-F1	76.20%
Test Accuracy	76.67%

Usage

import torch
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("roberta-base")
# Load model architecture first, then load state dict
model.load_state_dict(torch.load("model.pt", map_location="cpu"))
model.eval()

Architecture

• AHSP: Learnable layer-wise attention + gated pooling over transformer hidden states
• TPMB: Per-class memory bank with tail-aware contrastive prototype regularization
• LoRA: Low-rank adaptation on query/value projection matrices

Citation

If you use this model, please cite the HiPro-LoRA paper.