sgadagin/fine_tuned_sbert

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sgadagin/fine_tuned_sbert")
# Run inference
sentences = [
    'Security Event : Hack In Paris (16-17 June, 2011)\n\n\nHack In Paris is an international and corporate security event that will take place in Disneyland Paris® fromJune 16th to 17th of 2011. Please refer to the homepage to get up-to-date information about the event.\n\nTopics\nThe following list contains major topics the conference will cover. Please consider submitting even if the subject of your research is not listed here.\nAdvances in reverse engineering\nVulnerability research and exploitation\nPenetration testing and security assessment\nMalware analysis and new trends in malicous codes\nForensics, IT crime & law enforcement\nPrivacy issues: LOPPSI, HADOPI, …\nLow-level hacking (console security & mobile devices)\nRisk management and ISO 27001\nDates\nJanuary 20: CFP announced\nMarch 30: Submission deadline\nApril 15: Notification sent to authors\nApril 17: Program announcement\nJune 16-17: Hack In Paris\nJune 18: Nuit du Hack\nMore Information: https://hackinparis.com\n\n',
    'Security Event : Hack In Paris (16-17 June, 2011)\n\n\nHack In Paris is an international and corporate security event that will take place in Disneyland Paris® fromJune 16th to 17th of 2011. Please refer to the homepage to get up-to-date information about the event.\n\nTopics\nThe following list contains major topics the conference will cover. Please consider submitting even if the subject of your research is not listed here.\nAdvances in reverse engineering\nVulnerability research and exploitation\nPenetration testing and security assessment\nMalware analysis and new trends in malicous codes\nForensics, IT crime & law enforcement\nPrivacy issues: LOPPSI, HADOPI, …\nLow-level hacking (console security & mobile devices)\nRisk management and ISO 27001\nDates\nJanuary 20: CFP announced\nMarch 30: Submission deadline\nApril 15: Notification sent to authors\nApril 17: Program announcement\nJune 16-17: Hack In Paris\nJune 18: Nuit du Hack\nMore Information: https://hackinparis.com\n\n',
    'Google is going to shut down its social media network Google+ after the company suffered a massive data breach that exposed the private data of hundreds of thousands of Google Plus users to third-party developers.\n\nAccording to the tech giant, a security vulnerability in one of Google+\'s People APIs allowed third-party developers to access data for more than 500,000 users, including their usernames, email addresses, occupation, date of birth, profile photos, and gender-related information.\n\nSince Google+ servers do not keep API logs for more than two weeks, the company cannot confirm the number of users impacted by the vulnerability.\n\nHowever, Google assured its users that the company found no evidence that any developer was aware of this bug, or that the profile data was misused by any of the 438 developers that could have had access.\n"However, we ran a detailed analysis over the two weeks prior to patching the bug, and from that analysis, the Profiles of up to 500,000 Google+ accounts were potentially affected. Our analysis showed that up to 438 applications may have used this API," Google said in blog post published today.\nThe vulnerability was open since 2015 and fixed after Google discovered it in March 2018, but the company chose not to disclose the breach to the public—at the time when Facebook was being roasted for Cambridge Analytica scandal.\n\nThough Google has not revealed the technical details of the security vulnerability, the nature of the flaw seems to be something very similar to Facebook API flaw that recently allowed unauthorized developers to access private data from Facebook users.\n\nBesides admitting the security breach, Google also announced that the company is shutting down its social media network, acknowledging that Google+ failed to gain broad adoption or significant traction with consumers.\n"The consumer version of Google+ currently has low usage and engagement: 90 percent of Google+ user sessions are less than five seconds," Google said.\nIn response, the company has decided to shut down Google+ for consumers by the end of August 2019. However, Google+ will continue as a product for Enterprise users.\n\nGoogle Introduces New Privacy Controls Over Third-Party App Permissions\n\nAs part of its "Project Strobe," Google engineers also reviewed third-party developer access to Google account and Android device data; and has accordingly now introduced some new privacy controls.\n\nWhen a third-party app prompts users for access to their Google account data, clicking "Allow" button approves all requested permissions at once, leaving an opportunity for malicious apps to trick users into giving away powerful permissions.\nBut now Google has updated its Account Permissions system that asks for each requested permission individually rather than all at once, giving users more control over what type of account data they choose to share with each app.\n\nSince APIs can also allow developers to access users\' extremely sensitive data, like that of Gmail account, Google has limited access to Gmail API only for apps that directly enhance email functionality—such as email clients, email backup services and productivity services.\n\nGoogle shares fell over 2 percent to $1134.23 after the data breach reports.\n\n',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

• Size: 3,742 training samples
• Columns: sentence_0, sentence_1, and label
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	label
  type	string	string	int
  details	min: 37 tokens mean: 252.46 tokens max: 256 tokens	min: 37 tokens mean: 252.46 tokens max: 256 tokens	0: ~35.20% 1: ~10.30% 2: ~17.90% 3: ~36.60%

• Samples:

  sentence_0	sentence_1	label
  U.S. online fashion retailer SHEIN has admitted that the company has suffered a significant data breach after unknown hackers stole personally identifiable information (PII) of almost 6.5 million customers. Based in North Brunswick and founded in 2008, SHEIN has become one of the largest online fashion retailers that ships to more than 80 countries worldwide. The site has been initially designed to produce "affordable" and trendy fashion clothing for women. SHEIN revealed last weekend that its servers had been targeted by a "concerted criminal cyber-attack" that began in June this year and lasted until August 22, when the company was finally made aware of the potential theft. Soon after that, the company scanned its servers to remove all possible backdoored entry points, leveraging which hackers could again infiltrate the servers. SHEIN assured its customers that the website is now safe to visit. Hackers Stole Over 6.42 Million SHEIN Customers' Data Although details about the inci...	U.S. online fashion retailer SHEIN has admitted that the company has suffered a significant data breach after unknown hackers stole personally identifiable information (PII) of almost 6.5 million customers. Based in North Brunswick and founded in 2008, SHEIN has become one of the largest online fashion retailers that ships to more than 80 countries worldwide. The site has been initially designed to produce "affordable" and trendy fashion clothing for women. SHEIN revealed last weekend that its servers had been targeted by a "concerted criminal cyber-attack" that began in June this year and lasted until August 22, when the company was finally made aware of the potential theft. Soon after that, the company scanned its servers to remove all possible backdoored entry points, leveraging which hackers could again infiltrate the servers. SHEIN assured its customers that the website is now safe to visit. Hackers Stole Over 6.42 Million SHEIN Customers' Data Although details about the inci...	1
  A location based Social Networking platform with 45 million users,'Foursquare' was vulnerable to the primary email address disclosed. Foursquare is a Smartphone application that gives you details of nearby cafes, bars, shops, parks using GPS location and also tells about your friends nearby. According to a Penetration tester and hacker 'Jamal Eddine', an attacker can extract email addresses of all 45 million users just by using a few lines of scripting tool. Basically the flaw exists in the Invitation system of the Foursquare app. While testing the app, he found that invitation received on the recipient's end actually disclosing the sender's email address, as shown above. Invitation URL: https://foursquare.com/mehdi?action=acceptFriendship&expires=1378920415&src=wtbfe&uid=64761059&sig=mmlx96RwGrQ2fJAg4OWZhAWnDvc%3D Where 'uid' parameter represents the sender's profile ID. Hacker noticed that the parameter in the Invitation URL can be modified in order to spoof the sender profile i...	A location based Social Networking platform with 45 million users,'Foursquare' was vulnerable to the primary email address disclosed. Foursquare is a Smartphone application that gives you details of nearby cafes, bars, shops, parks using GPS location and also tells about your friends nearby. According to a Penetration tester and hacker 'Jamal Eddine', an attacker can extract email addresses of all 45 million users just by using a few lines of scripting tool. Basically the flaw exists in the Invitation system of the Foursquare app. While testing the app, he found that invitation received on the recipient's end actually disclosing the sender's email address, as shown above. Invitation URL: https://foursquare.com/mehdi?action=acceptFriendship&expires=1378920415&src=wtbfe&uid=64761059&sig=mmlx96RwGrQ2fJAg4OWZhAWnDvc%3D Where 'uid' parameter represents the sender's profile ID. Hacker noticed that the parameter in the Invitation URL can be modified in order to spoof the sender profile i...	1
  Earlier this week Dropbox team unveiled details of three critical vulnerabilities in Apple macOS operating system, which altogether could allow a remote attacker to execute malicious code on a targeted Mac computer just by convincing a victim into visiting a malicious web page. The reported vulnerabilities were originally discovered by Syndis, a cybersecurity firm hired by Dropbox to conduct simulated penetration testing attacks as Red Team on the company's IT infrastructure, including Apple software used by Dropbox. The vulnerabilities were discovered and disclosed to Apple security team in February this year, which were then patched by Apple just over one month later with the release of its March security updates. DropBox applauded Apple for its quick response to its bug report. According to DropBox, the vulnerabilities discovered by Syndis didn't just affect its macOS fleet, but also affected all Safari users running the latest version of the web browser and operating system at t...	Earlier this week Dropbox team unveiled details of three critical vulnerabilities in Apple macOS operating system, which altogether could allow a remote attacker to execute malicious code on a targeted Mac computer just by convincing a victim into visiting a malicious web page. The reported vulnerabilities were originally discovered by Syndis, a cybersecurity firm hired by Dropbox to conduct simulated penetration testing attacks as Red Team on the company's IT infrastructure, including Apple software used by Dropbox. The vulnerabilities were discovered and disclosed to Apple security team in February this year, which were then patched by Apple just over one month later with the release of its March security updates. DropBox applauded Apple for its quick response to its bug report. According to DropBox, the vulnerabilities discovered by Syndis didn't just affect its macOS fleet, but also affected all Safari users running the latest version of the web browser and operating system at t...	3

• Loss: SoftmaxLoss

Training Hyperparameters

Non-Default Hyperparameters

• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 8
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	Training Loss
1.0684	500	1.2186
2.1368	1000	1.145

Framework Versions

• Python: 3.12.9
• Sentence Transformers: 3.4.1
• Transformers: 4.49.0
• PyTorch: 2.6.0
• Accelerate: 1.4.0
• Datasets: 3.3.2
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers and SoftmaxLoss

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}