lm3_specimen_segmenter__birefnet_hr_swinl_1024

BiRefNet (Swin-L backbone) fine-tuned to separate plant material from everything else on a herbarium specimen sheet (binary foreground mask, used for background removal). Alternative to the LeafMachine3 default specimen_segmenter model (lm3_specimen_segmenter__unetpp_effb7_1024).

LM3 metadata

Field Value
LM3 action specimen_segmenter
LM3_default_model false
LM3_version 3.0.0 (loadable by the default stage; see Use in LeafMachine3 for the preprocessing caveat)
Model key birefnet_hr_swinl_1024 (training run birefnet-hr-1024-paperclean, checkpoint epoch 145)
Input whole sheet image, resized (not letterboxed) to 1024 x 1024, RGB, ImageNet mean/std normalization; tensor input [1, 3, 1024, 1024] float32
Output logits [1, 1, 1024, 1024]; apply sigmoid, resize to the sheet, threshold 0.5
Classes (1) plant (foreground)
Trained 2026-07-30

The ONNX uses the same tensor names and shapes as the default UNet++ export.

Architecture

Component Detail
Network BiRefNet (bilateral reference): localization module plus a reconstruction decoder guided by in-image and gradient references, with deformable-convolution ASPP blocks
Backbone Swin Transformer Large (swin_v1_l)
Starting weights BiRefNet_HR-general-epoch_130 (general high-resolution background removal, trained at 2048 x 2048)
Fine-tuning full network, at 1024 x 1024
Output multi-scale predictions during training; the final full-resolution map at inference

Credits

  • BiRefNet (Peng Zheng et al., MIT): architecture, training code and the BiRefNet_HR-general weights this model starts from. https://github.com/ZhengPeng7/BiRefNet
  • Zheng et al., "Bilateral Reference for High-Resolution Dichotomous Image Segmentation", CAAI Artificial Intelligence Research, 2024.
  • Swin Transformer: Liu et al., "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows", 2021.
  • SAM 3 (Meta): produced the first-generation masks the training targets descend from (see Training data).

Files

Folder File Use
pytorch/ epoch_145.pth BiRefNet state dict; load with the BiRefNet repo (swin_v1_l, size 1024); retraining
onnx/ model.onnx onnxruntime, opset 17; deformable convolutions decomposed into standard ops (no custom ops)
docs/ compare_*.jpg example panels (see Examples)

SHA-256 of every file is in manifest.json. There is no CoreML export: coremltools cannot convert BiRefNet's deformable convolutions.

Training data

There is no hand-drawn ground truth for this task. The training targets are machine-generated masks, refined in three steps:

  1. Boxes to masks. Specimen boxes from the LeafMachine3 plant detection set prompt SAM 3; the per-box masks are unioned, holes filled, and failed masks gated out. 21,098 sheets kept.
  2. Self-training. A first UNet++ trained on those masks re-predicts every herbarium sheet; its cleaner predictions replace the SAM 3 masks. 2,193 field photographs are excluded as out of domain. 18,904 sheets.
  3. Paper removal. A color rule samples the mounting paper around each mask and deletes large in-mask regions that match it.

The result is dataset LM3_Specimen_Segmentation-specimen-unetMasksFromSam3-paperclean: 18,904 sheet images from 75 labeling projects, one plant class. Images and masks are not distributed.

Split Images
train 15,154
val 1,910
test 1,840

Training

Fine-tuned from epoch 130 to 145 (15 epochs) on the train split, batch 4, 1024 x 1024, learning rate 1e-5, BiRefNet's background-removal loss preset (binary cross-entropy 30, IoU 0.5, MAE 100, SSIM 10), with the contour-friendly loss switch over the final epochs.

Evaluation

Scores are agreement with the machine-generated masks described above, not with human annotation. They measure how well the model reproduces its training targets on held-out sheets; a model that corrects a flawed target is penalized for it. Read them as a consistency check and judge quality from the examples. Every LeafMachine3 specimen-segmenter card is scored on the same random 150-sheet sample of each split, so the numbers are comparable across models.

Split Sheets scored Dice (mean) Dice (median) IoU (mean)
val, exported ONNX (earlier antialiased resize) 150 of 1,910 0.951 0.983 0.925
test, LM3 production path (training resize) 150 of 1,840 0.946 0.983 0.921

Scored at native sheet resolution; 3% of test sheets score below 0.5 Dice. The test row uses the model's training resize (LeafMachine3's production path); the val row predates it and used an antialiased resize, which changes the masks very little (mean IoU 0.99 between the two). ONNX parity with the PyTorch checkpoint: mask IoU 1.0000 (max probability difference 9.9e-05) on held-out sheets.

Examples

Background removal on the same sheets by this model (BiRefNet) and by the other LeafMachine3 specimen segmenters trained on the same data, for comparison. Black marks what each model removed; the percentage is the share of the sheet kept. Raw model masks, before LeafMachine3's paper-removal follow-up. File names give each sheet's dataset split. Panels 1 to 12 are a standing comparison set that includes deliberately hard sheets (several from the training split); panels 13 to 20 are random held-out test sheets.


Acer saccharum, CMN_1804514859 路 test split 路 kept: UNet++ 40.1%, BiRefNet 40.2%, YOLO26x-seg 47.1%

Acer saccharum, CM_2859177966 路 train split 路 kept: UNet++ 28.5%, BiRefNet 28.5%, YOLO26x-seg 32.4%

Cassia fistula, FLAS_1457816931 路 val split 路 kept: UNet++ 23.0%, BiRefNet 23.3%, YOLO26x-seg 23.8%

Chrysophyllum gonocarpum, NY_1928830727 路 val split 路 kept: UNet++ 22.6%, BiRefNet 22.2%, YOLO26x-seg 30.8%

CHC21_009_9 路 val split 路 kept: UNet++ 2.6%, BiRefNet 2.6%, YOLO26x-seg 2.7%

Macaranga saccifera, MNHN_730851102 路 val split 路 kept: UNet++ 44.4%, BiRefNet 44.6%, YOLO26x-seg 49.3%

Macaranga magnifolia, MNHN_731821562 路 train split 路 kept: UNet++ 11.0%, BiRefNet 0.5%, YOLO26x-seg 47.6%

Macaranga gracilis, UM_1807486051 路 train split 路 kept: UNet++ 31.2%, BiRefNet 34.0%, YOLO26x-seg 37.8%

Torricellia angulata, K_912592861 路 train split 路 kept: UNet++ 22.7%, BiRefNet 24.2%, YOLO26x-seg 27.5%

Quercus grisebachii, CLF_1437883182 路 train split 路 kept: UNet++ 22.5%, BiRefNet 23.5%, YOLO26x-seg 23.8%

Quercus semilanuginosa, CLF_1437893278 路 train split 路 kept: UNet++ 20.5%, BiRefNet 32.1%, YOLO26x-seg 29.5%

Quercus pinnatiloba, MNHN_731018123 路 train split 路 kept: UNet++ 33.0%, BiRefNet 32.7%, YOLO26x-seg 33.7%

Lonicera maackii, KYO_2443051967 路 test split 路 kept: UNet++ 15.1%, BiRefNet 15.5%, YOLO26x-seg 19.2%

Macaranga waturandangii, BO1284103_4067113860 路 test split 路 kept: UNet++ 33.8%, BiRefNet 34.1%, YOLO26x-seg 35.6%

Melanophylla alnifolia, L_2517447339 路 test split 路 kept: UNet++ 22.2%, BiRefNet 21.9%, YOLO26x-seg 23.7%

Raphanus raphanistrum, ASU_2270499210 路 test split 路 kept: UNet++ 10.0%, BiRefNet 10.2%, YOLO26x-seg 15.7%

Posoqueria longiflora, UM_1807432107 路 test split 路 kept: UNet++ 29.4%, BiRefNet 29.5%, YOLO26x-seg 31.4%

1D41D09F-BC60-4E11-9283-3A0F23787BA7_1_105_c 路 test split 路 kept: UNet++ 24.9%, BiRefNet 24.9%, YOLO26x-seg 27.8%

Piper salinasanum, MNHN_438671634 路 test split 路 kept: UNet++ 12.1%, BiRefNet 12.0%, YOLO26x-seg 13.3%

lab_tilia_tomentosa_ny1110-08-4 路 test split 路 kept: UNet++ 7.5%, BiRefNet 7.5%, YOLO26x-seg 7.7%

Use in LeafMachine3

From the LM3 release that adds model keys to the specimen_segmenter stage (version TBD), the stage feeds this model its exact training input, a plain linear resize to 1024 x 1024, then bilinear-upsamples the probabilities and thresholds. Install and select it:

lm3 models install --model specimen_segmenter=birefnet_hr_swinl_1024
modules:
  specimen_segmenter:
    model: { key: "birefnet_hr_swinl_1024", path: "models/specimen_segmenter/birefnet_hr_swinl_1024/model.onnx", format: "onnx" }
    conf: 0.5
    paperclean: true

LM3 3.0.0, before model keys, can also load this ONNX by pointing model.path at it, but feeds it a letterboxed sheet; masks then match the training-style input at mean IoU 0.97. The ONNX is about four times the size of the default's and markedly slower on CPU; a GPU is recommended.

from huggingface_hub import hf_hub_download
path = hf_hub_download("phyloforfun/lm3_specimen_segmenter__birefnet_hr_swinl_1024", "onnx/model.onnx")

Citation

Weaver, W. N. (2026) LeafMachine3. https://leafmachine.org

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 馃檵 Ask for provider support