README.md

metadata

language: en
license: apache-2.0
tags:
  - aqea
  - compression
  - embeddings
  - similarity-search
  - vector-database
datasets:
  - mteb/stsbenchmark-sts
base_model: intfloat/e5-large-v2

AQEA: aqea-e5-large-v2-29x

E5-Large text embeddings compressed 29x while preserving 98.2% similarity ranking

📊 Performance

Metric	Value
Compression Ratio	29.3x
Spearman ρ	98.2%
Source Dimension	1024D
Compressed Dimension	35D
Storage Savings	96.6%

🚀 Usage

from aqea import AQEACompressor

# Load pre-trained compressor
compressor = AQEACompressor.from_pretrained("nextxag/aqea-e5-large-v2-29x")

# Compress embeddings
embeddings = model.encode(texts)  # 1024D
compressed = compressor.compress(embeddings)  # 35D

# Decompress for retrieval
reconstructed = compressor.decompress(compressed)  # 1024D

📁 Files

• weights.aqwt - Binary weights (AQEA native format)
• config.json - Model configuration

🔬 How It Works

AQEA (Adaptive Quantized Embedding Architecture) uses learned linear projections with Pre-Quantify rotation to compress embeddings while maximally preserving pairwise similarity rankings (measured by Spearman correlation).

📚 Citation

@software{aqea2024,
  title = {AQEA: Adaptive Quantized Embedding Architecture},
  author = {AQEA Team},
  year = {2024},
  url = {https://huggingface.co/nextxag}
}

📄 License

Apache 2.0