Instructions to use phyloforfun/lm3_specimen_segmenter__birefnet_hr_swinl_1024 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- BiRefNet
How to use phyloforfun/lm3_specimen_segmenter__birefnet_hr_swinl_1024 with BiRefNet:
# Option 1: use with transformers from transformers import AutoModelForImageSegmentation birefnet = AutoModelForImageSegmentation.from_pretrained("phyloforfun/lm3_specimen_segmenter__birefnet_hr_swinl_1024", trust_remote_code=True)# Option 2: use with BiRefNet # Install from https://github.com/ZhengPeng7/BiRefNet from models.birefnet import BiRefNet model = BiRefNet.from_pretrained("phyloforfun/lm3_specimen_segmenter__birefnet_hr_swinl_1024") - Notebooks
- Google Colab
- Kaggle
lm3_specimen_segmenter__birefnet_hr_swinl_1024
BiRefNet (Swin-L backbone) fine-tuned to separate plant material from everything else on a herbarium specimen sheet (binary foreground mask, used for background removal). Alternative to the LeafMachine3 default specimen_segmenter model (lm3_specimen_segmenter__unetpp_effb7_1024).
LM3 metadata
| Field | Value |
|---|---|
| LM3 action | specimen_segmenter |
| LM3_default_model | false |
| LM3_version | 3.0.0 (loadable by the default stage; see Use in LeafMachine3 for the preprocessing caveat) |
| Model key | birefnet_hr_swinl_1024 (training run birefnet-hr-1024-paperclean, checkpoint epoch 145) |
| Input | whole sheet image, resized (not letterboxed) to 1024 x 1024, RGB, ImageNet mean/std normalization; tensor input [1, 3, 1024, 1024] float32 |
| Output | logits [1, 1, 1024, 1024]; apply sigmoid, resize to the sheet, threshold 0.5 |
| Classes (1) | plant (foreground) |
| Trained | 2026-07-30 |
The ONNX uses the same tensor names and shapes as the default UNet++ export.
Architecture
| Component | Detail |
|---|---|
| Network | BiRefNet (bilateral reference): localization module plus a reconstruction decoder guided by in-image and gradient references, with deformable-convolution ASPP blocks |
| Backbone | Swin Transformer Large (swin_v1_l) |
| Starting weights | BiRefNet_HR-general-epoch_130 (general high-resolution background removal, trained at 2048 x 2048) |
| Fine-tuning | full network, at 1024 x 1024 |
| Output | multi-scale predictions during training; the final full-resolution map at inference |
Credits
- BiRefNet (Peng Zheng et al., MIT): architecture, training code and the
BiRefNet_HR-generalweights this model starts from. https://github.com/ZhengPeng7/BiRefNet - Zheng et al., "Bilateral Reference for High-Resolution Dichotomous Image Segmentation", CAAI Artificial Intelligence Research, 2024.
- Swin Transformer: Liu et al., "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows", 2021.
- SAM 3 (Meta): produced the first-generation masks the training targets descend from (see Training data).
Files
| Folder | File | Use |
|---|---|---|
pytorch/ |
epoch_145.pth |
BiRefNet state dict; load with the BiRefNet repo (swin_v1_l, size 1024); retraining |
onnx/ |
model.onnx |
onnxruntime, opset 17; deformable convolutions decomposed into standard ops (no custom ops) |
docs/ |
compare_*.jpg |
example panels (see Examples) |
SHA-256 of every file is in manifest.json. There is no CoreML export: coremltools cannot convert BiRefNet's deformable convolutions.
Training data
There is no hand-drawn ground truth for this task. The training targets are machine-generated masks, refined in three steps:
- Boxes to masks.
Specimenboxes from the LeafMachine3 plant detection set prompt SAM 3; the per-box masks are unioned, holes filled, and failed masks gated out. 21,098 sheets kept. - Self-training. A first UNet++ trained on those masks re-predicts every herbarium sheet; its cleaner predictions replace the SAM 3 masks. 2,193 field photographs are excluded as out of domain. 18,904 sheets.
- Paper removal. A color rule samples the mounting paper around each mask and deletes large in-mask regions that match it.
The result is dataset LM3_Specimen_Segmentation-specimen-unetMasksFromSam3-paperclean: 18,904 sheet images from 75 labeling projects, one plant class. Images and masks are not distributed.
| Split | Images |
|---|---|
| train | 15,154 |
| val | 1,910 |
| test | 1,840 |
Training
Fine-tuned from epoch 130 to 145 (15 epochs) on the train split, batch 4, 1024 x 1024, learning rate 1e-5, BiRefNet's background-removal loss preset (binary cross-entropy 30, IoU 0.5, MAE 100, SSIM 10), with the contour-friendly loss switch over the final epochs.
Evaluation
Scores are agreement with the machine-generated masks described above, not with human annotation. They measure how well the model reproduces its training targets on held-out sheets; a model that corrects a flawed target is penalized for it. Read them as a consistency check and judge quality from the examples. Every LeafMachine3 specimen-segmenter card is scored on the same random 150-sheet sample of each split, so the numbers are comparable across models.
| Split | Sheets scored | Dice (mean) | Dice (median) | IoU (mean) |
|---|---|---|---|---|
| val, exported ONNX (earlier antialiased resize) | 150 of 1,910 | 0.951 | 0.983 | 0.925 |
| test, LM3 production path (training resize) | 150 of 1,840 | 0.946 | 0.983 | 0.921 |
Scored at native sheet resolution; 3% of test sheets score below 0.5 Dice. The test row uses the model's training resize (LeafMachine3's production path); the val row predates it and used an antialiased resize, which changes the masks very little (mean IoU 0.99 between the two). ONNX parity with the PyTorch checkpoint: mask IoU 1.0000 (max probability difference 9.9e-05) on held-out sheets.
Examples
Background removal on the same sheets by this model (BiRefNet) and by the other LeafMachine3 specimen segmenters trained on the same data, for comparison. Black marks what each model removed; the percentage is the share of the sheet kept. Raw model masks, before LeafMachine3's paper-removal follow-up. File names give each sheet's dataset split. Panels 1 to 12 are a standing comparison set that includes deliberately hard sheets (several from the training split); panels 13 to 20 are random held-out test sheets.
![]() Acer saccharum, CMN_1804514859 路 test split 路 kept: UNet++ 40.1%, BiRefNet 40.2%, YOLO26x-seg 47.1% |
![]() Acer saccharum, CM_2859177966 路 train split 路 kept: UNet++ 28.5%, BiRefNet 28.5%, YOLO26x-seg 32.4% |
![]() Cassia fistula, FLAS_1457816931 路 val split 路 kept: UNet++ 23.0%, BiRefNet 23.3%, YOLO26x-seg 23.8% |
![]() Chrysophyllum gonocarpum, NY_1928830727 路 val split 路 kept: UNet++ 22.6%, BiRefNet 22.2%, YOLO26x-seg 30.8% |
![]() CHC21_009_9 路 val split 路 kept: UNet++ 2.6%, BiRefNet 2.6%, YOLO26x-seg 2.7% |
![]() Macaranga saccifera, MNHN_730851102 路 val split 路 kept: UNet++ 44.4%, BiRefNet 44.6%, YOLO26x-seg 49.3% |
![]() Macaranga magnifolia, MNHN_731821562 路 train split 路 kept: UNet++ 11.0%, BiRefNet 0.5%, YOLO26x-seg 47.6% |
![]() Macaranga gracilis, UM_1807486051 路 train split 路 kept: UNet++ 31.2%, BiRefNet 34.0%, YOLO26x-seg 37.8% |
![]() Torricellia angulata, K_912592861 路 train split 路 kept: UNet++ 22.7%, BiRefNet 24.2%, YOLO26x-seg 27.5% |
![]() Quercus grisebachii, CLF_1437883182 路 train split 路 kept: UNet++ 22.5%, BiRefNet 23.5%, YOLO26x-seg 23.8% |
![]() Quercus semilanuginosa, CLF_1437893278 路 train split 路 kept: UNet++ 20.5%, BiRefNet 32.1%, YOLO26x-seg 29.5% |
![]() Quercus pinnatiloba, MNHN_731018123 路 train split 路 kept: UNet++ 33.0%, BiRefNet 32.7%, YOLO26x-seg 33.7% |
![]() Lonicera maackii, KYO_2443051967 路 test split 路 kept: UNet++ 15.1%, BiRefNet 15.5%, YOLO26x-seg 19.2% |
![]() Macaranga waturandangii, BO1284103_4067113860 路 test split 路 kept: UNet++ 33.8%, BiRefNet 34.1%, YOLO26x-seg 35.6% |
![]() Melanophylla alnifolia, L_2517447339 路 test split 路 kept: UNet++ 22.2%, BiRefNet 21.9%, YOLO26x-seg 23.7% |
![]() Raphanus raphanistrum, ASU_2270499210 路 test split 路 kept: UNet++ 10.0%, BiRefNet 10.2%, YOLO26x-seg 15.7% |
![]() Posoqueria longiflora, UM_1807432107 路 test split 路 kept: UNet++ 29.4%, BiRefNet 29.5%, YOLO26x-seg 31.4% |
![]() 1D41D09F-BC60-4E11-9283-3A0F23787BA7_1_105_c 路 test split 路 kept: UNet++ 24.9%, BiRefNet 24.9%, YOLO26x-seg 27.8% |
![]() Piper salinasanum, MNHN_438671634 路 test split 路 kept: UNet++ 12.1%, BiRefNet 12.0%, YOLO26x-seg 13.3% |
![]() lab_tilia_tomentosa_ny1110-08-4 路 test split 路 kept: UNet++ 7.5%, BiRefNet 7.5%, YOLO26x-seg 7.7% |
Use in LeafMachine3
From the LM3 release that adds model keys to the specimen_segmenter stage (version TBD), the stage feeds this model its exact training input, a plain linear resize to 1024 x 1024, then bilinear-upsamples the probabilities and thresholds. Install and select it:
lm3 models install --model specimen_segmenter=birefnet_hr_swinl_1024
modules:
specimen_segmenter:
model: { key: "birefnet_hr_swinl_1024", path: "models/specimen_segmenter/birefnet_hr_swinl_1024/model.onnx", format: "onnx" }
conf: 0.5
paperclean: true
LM3 3.0.0, before model keys, can also load this ONNX by pointing model.path at it, but feeds it a letterboxed sheet; masks then match the training-style input at mean IoU 0.97. The ONNX is about four times the size of the default's and markedly slower on CPU; a GPU is recommended.
from huggingface_hub import hf_hub_download
path = hf_hub_download("phyloforfun/lm3_specimen_segmenter__birefnet_hr_swinl_1024", "onnx/model.onnx")
Citation
Weaver, W. N. (2026) LeafMachine3. https://leafmachine.org



















