nanos-hpe/bge-small-qs

SentenceTransformer based on BAAI/bge-small-en

This is a sentence-transformers model finetuned from BAAI/bge-small-en. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-small-en
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 384 tokens
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'What is the website to find services for customers purchasing from a commercial reseller?',
    'Parts and Materials\nHPE will provide HPE-supported replacement parts and materials necessary to maintain the covered hardware\nproduct in operating condition, including parts and materials for available and recommended engineering\nimprovements. \xa0\nParts and components that have reached their maximum supported lifetime and/or the maximum usage\nlimitations as set forth in the manufacturer\'s operating manual, product quick-specs, or the technical product\ndata sheet will not be provided, repaired, or replaced as part of these services.\n\xa0\nHow to Purchase Services\nServices are sold by Hewlett Packard Enterprise and Hewlett Packard Enterprise Authorized Service Partners:\nServices for customers purchasing from HPE or an enterprise reseller are quoted using HPE order\nconfiguration tools.\nCustomers purchasing from a commercial reseller can find services at\nhttps://ssc.hpe.com/portal/site/ssc/\n\xa0\nAI Powered and Digitally Enabled Support Experience\nAchieve faster time to resolution with access to product-specific resources and expertise through a digital and\ndata driven customer experience \xa0\nSign into the HPE Support Center experience, featuring streamlined self-serve case creation and\nmanagement capabilities with inline knowledge recommendations. You will also find personalized task alerts\nand powerful troubleshooting support through an intelligent virtual agent with seamless transition when needed\nto a live support agent. \xa0\nhttps://support.hpe.com/hpesc/public/home/signin\nConsume IT On Your Terms\nHPE GreenLake  edge-to-cloud platform brings the cloud experience directly to your apps and data wherever\nthey are-the edge, colocations, or your data center. It delivers cloud services for on-premises IT infrastructure\nspecifically tailored to your most demanding workloads. With a pay-per-use, scalable, point-and-click self-\nservice experience that is managed for you, HPE GreenLake edge-to-cloud platform accelerates digital\ntransformation in a distributed, edge-to-cloud world.\nGet faster time to market\nSave on TCO, align costs to business\nScale quickly, meet unpredictable demand\nSimplify IT operations across your data centers and clouds\nTo learn more about HPE Services, please contact your Hewlett Packard Enterprise sales representative or\nHewlett Packard Enterprise Authorized Channel Partner. \xa0 Contact information for a representative in your area\ncan be found at "Contact HPE"  https://www.hpe.com/us/en/contact-hpe.html \xa0\nFor more information\nhttp://www.hpe.com/services\nQuickSpecs\nHPE Cray XD675\nService and Support\nDA - 17239\xa0\xa0\xa0Worldwide QuickSpecs — Version 4 — 8/19/2024\nPage\xa0 13',
    'HPE Cray XD675 Server Top View\nItem Description \xa0 \xa0\n1. 8x AMD MI300X OAM Accelerator \xa0 \xa0\n\xa0\nQuickSpecs\nHPE Cray XD675\nOverview\nDA - 17239\xa0\xa0\xa0Worldwide QuickSpecs — Version 4 — 8/19/2024\nPage\xa0 2',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.4857
cosine_accuracy@3	0.8048
cosine_accuracy@5	0.8619
cosine_accuracy@10	0.9095
cosine_precision@1	0.4857
cosine_precision@3	0.2683
cosine_precision@5	0.1724
cosine_precision@10	0.091
cosine_recall@1	0.4857
cosine_recall@3	0.8048
cosine_recall@5	0.8619
cosine_recall@10	0.9095
cosine_ndcg@10	0.7184
cosine_mrr@10	0.6552
cosine_map@100	0.6599
dot_accuracy@1	0.4857
dot_accuracy@3	0.8048
dot_accuracy@5	0.8619
dot_accuracy@10	0.9095
dot_precision@1	0.4857
dot_precision@3	0.2683
dot_precision@5	0.1724
dot_precision@10	0.091
dot_recall@1	0.4857
dot_recall@3	0.8048
dot_recall@5	0.8619
dot_recall@10	0.9095
dot_ndcg@10	0.7184
dot_mrr@10	0.6552
dot_map@100	0.6599

Training Details

Training Dataset

Unnamed Dataset

• Size: 3,221 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 3 tokens mean: 22.72 tokens max: 80 tokens	min: 36 tokens mean: 328.94 tokens max: 512 tokens

• Samples:

  sentence_0	sentence_1
  What is the maximum number of Apollo n2X00 series chassis that can fit in a 42U rack?	HPE Apollo 2000 Gen10 Plus System   HPE is bringing the power of supercomputing to datacenters of any size with the Apollo 2000 Gen10 Plus system.   The HPE Apollo 2000 Gen10 Plus System is a dense, multi-server platform that packs incredible performance and workload flexibility into a small datacenter space, while delivering the efficiencies of a shared infrastructure. It is designed to provide a bridge to scale-out architecture for traditional data centers, so enterprise and SME customers can achieve the space-saving value of density-optimized infrastructure in a cost-effective and non-disruptive manner.   The Apollo 2000 Gen10 Plus offers a density optimized, shared infrastructure with a flexible scale-out architecture to support a variety of workloads from remote site systems to large HPC clusters and everything in between. HPE iLO5 provides built-in firmware-level server security with silicon root of trust. It can be deployed cost-effectively starting with a single 2U, shared infrastructure chassis and configured with a variety of storage options to meet the configuration needs of a wide variety of scale-out workloads.   The Apollo 2000 Gen10 Plus System delivers up to four times the density of a traditional rack mount server with up to four ProLiant Gen10 Plus independent servers per 2U mounted in standard racks with rear-aisle serviceability access. A 42U rack fits up to 20 Apollo n2X00 series chassis accommodating up to 80 servers per rack. What's New   Support for up to four Xilinx Alveo U50 single wide  GPU's  in XL290n node. Enables a robust stack of Intel 3 rd generation Xeon Scalable Processors to increase your power density and increase datacenter efficiency. Intel AVX-512 * feature increases memory bandwidth, improves frequency management to enable greater performance. Also Speed Select Technology (SST) allows Core count and frequency flexibility * The Direct Liquid Cooling (DLC) option for the Apollo 2000 Gen10 Plus System comes ready to plug and play. Choose from either CPU only or CPU plus memory cooling options. Enables flexible choices with Intel 3 rd Generation Xeon Scalable Processors and AMD 2 nd and 3 rd generation EPYC Processors New flexible infrastructure offers multiple storage options, 8 memory channels and 3200 MT/s memory, PCIe Gen4 and support for processors over 250W for improved application performance. Complete software portfolio for all customer workloads, for node to rack management, including comprehensive integrated cluster management software Secure from the start with firmware anchored into silicon with iLO5 and silicon root of trust for the highest level of system security   Notes: *Available on select processors   QuickSpecs HPE Apollo 2000 Gen10 Plus System Overview DA - 16526   Worldwide QuickSpecs — Version 41 — 7/1/2024 Page  1
  What is the maximum number of independent servers that can be mounted in a single 2U Apollo 2000 Gen10 Plus System chassis?	HPE Apollo 2000 Gen10 Plus System   HPE is bringing the power of supercomputing to datacenters of any size with the Apollo 2000 Gen10 Plus system.   The HPE Apollo 2000 Gen10 Plus System is a dense, multi-server platform that packs incredible performance and workload flexibility into a small datacenter space, while delivering the efficiencies of a shared infrastructure. It is designed to provide a bridge to scale-out architecture for traditional data centers, so enterprise and SME customers can achieve the space-saving value of density-optimized infrastructure in a cost-effective and non-disruptive manner.   The Apollo 2000 Gen10 Plus offers a density optimized, shared infrastructure with a flexible scale-out architecture to support a variety of workloads from remote site systems to large HPC clusters and everything in between. HPE iLO5 provides built-in firmware-level server security with silicon root of trust. It can be deployed cost-effectively starting with a single 2U, shared infrastructure chassis and configured with a variety of storage options to meet the configuration needs of a wide variety of scale-out workloads.   The Apollo 2000 Gen10 Plus System delivers up to four times the density of a traditional rack mount server with up to four ProLiant Gen10 Plus independent servers per 2U mounted in standard racks with rear-aisle serviceability access. A 42U rack fits up to 20 Apollo n2X00 series chassis accommodating up to 80 servers per rack. What's New   Support for up to four Xilinx Alveo U50 single wide  GPU's  in XL290n node. Enables a robust stack of Intel 3 rd generation Xeon Scalable Processors to increase your power density and increase datacenter efficiency. Intel AVX-512 * feature increases memory bandwidth, improves frequency management to enable greater performance. Also Speed Select Technology (SST) allows Core count and frequency flexibility * The Direct Liquid Cooling (DLC) option for the Apollo 2000 Gen10 Plus System comes ready to plug and play. Choose from either CPU only or CPU plus memory cooling options. Enables flexible choices with Intel 3 rd Generation Xeon Scalable Processors and AMD 2 nd and 3 rd generation EPYC Processors New flexible infrastructure offers multiple storage options, 8 memory channels and 3200 MT/s memory, PCIe Gen4 and support for processors over 250W for improved application performance. Complete software portfolio for all customer workloads, for node to rack management, including comprehensive integrated cluster management software Secure from the start with firmware anchored into silicon with iLO5 and silicon root of trust for the highest level of system security   Notes: *Available on select processors   QuickSpecs HPE Apollo 2000 Gen10 Plus System Overview DA - 16526   Worldwide QuickSpecs — Version 41 — 7/1/2024 Page  1
  What is the processor type supported by the HPE Apollo n2800 Gen10 Plus 24 SFF Flexible CTO chassis?	HPE Apollo n2600 Gen10 Plus SFF CTO Chassis supports both Intel and AMD based server nodes HPE Apollo n2800 Gen10 Plus 24 SFF Flexible CTO chassis supports Intel based server nodes Backplane selection determines number and type of drives supported Item Description Item Description 1 SFF hot-plug drives 3 Health LED 2 Serial number/iLO information pull tab 4 UID button LED   Chassis Rear Panel Components - 4 x 1U nodes Item Description Item Description 1 Server 3 & 4 4 iLO Ports 2 HPE Apollo Platform Manager (APM) 2.0 port 5 Server 1 & 2 3 Power supply 1 & 2 6 Optional Rack Consolidation Module (RCM)     QuickSpecs HPE Apollo 2000 Gen10 Plus System Overview DA - 16526   Worldwide QuickSpecs — Version 41 — 7/1/2024 Page  2

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 256
• per_device_eval_batch_size: 256
• num_train_epochs: 20
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 256
• per_device_eval_batch_size: 256
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 20
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	cosine_map@100
1.0	7	0.4864
2.0	14	0.5209
3.0	21	0.5131
4.0	28	0.5047
5.0	35	0.5480
6.0	42	0.5808
7.0	49	0.5950
7.1429	50	0.5975
8.0	56	0.6145
9.0	63	0.6268
10.0	70	0.6292
11.0	77	0.6385
12.0	84	0.6445
13.0	91	0.6279
14.0	98	0.6296
14.2857	100	0.6321
15.0	105	0.6317
16.0	112	0.6401
17.0	119	0.6590
18.0	126	0.6562
19.0	133	0.6599

Framework Versions

• Python: 3.11.8
• Sentence Transformers: 3.1.1
• Transformers: 4.45.2
• PyTorch: 2.2.2+cu121
• Accelerate: 1.1.1
• Datasets: 3.1.0
• Tokenizers: 0.20.3

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}