PAWBench: How Far Are We from Probabilistically Aligned World Modeling?
Paper • 2608.27345 • Published • 39
kernels-community
Team
community
AI & ML interests
None defined yet.
Recent Activity
View all activity
Team members 4 private
sayakpaul
authored a
paper about 2 months ago
sayakpaul
submitted a
paper to Daily Papers about 2 months ago
sayakpaul
authored 4
papers 3 months ago
DynEval: Holistic Evaluations of T2I Generative Models in the Wild
Paper • 2607.11199 • Published • 1
Posterior Augmented Flow Matching
Paper • 2605.00825 • Published
4KLSDB: A Large-Scale Dataset for 4K Image Restoration and Generation
Paper • 2605.24762 • Published • 1
Flash-BoN: Instant Drafts for Inference-Time Scaling in Diffusion Models
Paper • 2607.04461 • Published • 11
sayakpaul
submitted a
paper to Daily Papers 3 months ago
Post
261
We have recently added Torch Stable ABI support to kernels and kernel-builder. This allows kernel developers to target a particular Torch version and the kernel will be supported on that Torch version and later Torch versions (up to ~2 years).
This makes it much easier to write kernels with long-term support and not just the last two Torch releases.
We have also started rolling out Stable ABI support to kernels in kernels-community, starting with Flash Attention 3, supporting Torch 2.9 and later as well as CUDA versions starting at 12.6:
https://huggingface.co/kernels/kernels-community/flash-attn3/tree/v1/build
This makes it much easier to write kernels with long-term support and not just the last two Torch releases.
We have also started rolling out Stable ABI support to kernels in kernels-community, starting with Flash Attention 3, supporting Torch 2.9 and later as well as CUDA versions starting at 12.6:
https://huggingface.co/kernels/kernels-community/flash-attn3/tree/v1/build
- 1 reply
Post
297
Two large changes in kernel-builder this week:
kernel-builder now links libstdc++ dynamically. To support a wide range of systems, we build against libstdc++ from manylinux_2_28 (EL 8 and later).
Following our Torch support policy that the current and previous Torch versions are supported, Torch 2.10 support was removed. We will soon also support the Torch stable ABI, so that it is possible to write kernels that support a large number of Torch versions.
kernel-builder now links libstdc++ dynamically. To support a wide range of systems, we build against libstdc++ from manylinux_2_28 (EL 8 and later).
Following our Torch support policy that the current and previous Torch versions are supported, Torch 2.10 support was removed. We will soon also support the Torch stable ABI, so that it is possible to write kernels that support a large number of Torch versions.
sayakpaul
authored 2
papers 8 months ago
sayakpaul
submitted a
paper to Daily Papers 8 months ago
sayakpaul
authored a
paper 8 months ago
Post
2,849
kernels 0.12 is out! 🎉
Changes:
* Support for kernel version branches to gracefully roll out kernel API changes.
* Support for PyTorch 2.10.
* kernel-builder is now merged into the kernels repo.
* Initial support for standardized kernel benchmarks.
https://github.com/huggingface/kernels/releases/tag/v0.12.0
Changes:
* Support for kernel version branches to gracefully roll out kernel API changes.
* Support for PyTorch 2.10.
* kernel-builder is now merged into the kernels repo.
* Initial support for standardized kernel benchmarks.
https://github.com/huggingface/kernels/releases/tag/v0.12.0
Post
564
We have released
Headline features:
* 🔮 Supports building kernels for the brand-new PyTorch 2.9.0.
* 🪟 Experimental support for building Windows kernels.
kernel-builder 0.7.0: https://github.com/huggingface/kernel-builder/releases/tag/v0.7.0Headline features:
* 🔮 Supports building kernels for the brand-new PyTorch 2.9.0.
* 🪟 Experimental support for building Windows kernels.
sayakpaul
authored a
paper about 1 year ago
Post
3,564
Fast LoRA inference for Flux with Diffusers and PEFT 🚨
There are great materials that demonstrate how to optimize inference for popular image generation models, such as Flux. However, very few cover how to serve LoRAs fast, despite LoRAs being an inseparable part of their adoption.
In our latest post, @BenjaminB and I show different techniques to optimize LoRA inference for the Flux family of models for image generation. Our recipe includes the use of:
1.
2. Flash Attention 3 (when compatible)
3. Dynamic FP8 weight quantization (when compatible)
4. Hotswapping for avoiding recompilation during swapping new LoRAs 🤯
We have tested our recipe with Flux.1-Dev on both H100 and RTX 4090. We achieve at least a *2x speedup* in either of the GPUs. We believe our recipe is grounded in the reality of how LoRA-based use cases are generally served. So, we hope this will be beneficial to the community 🤗
Even though our recipe was tested primarily with NVIDIA GPUs, it should also work with AMD GPUs.
Learn the details and the full code here:
https://huggingface.co/blog/lora-fast
There are great materials that demonstrate how to optimize inference for popular image generation models, such as Flux. However, very few cover how to serve LoRAs fast, despite LoRAs being an inseparable part of their adoption.
In our latest post, @BenjaminB and I show different techniques to optimize LoRA inference for the Flux family of models for image generation. Our recipe includes the use of:
1.
torch.compile2. Flash Attention 3 (when compatible)
3. Dynamic FP8 weight quantization (when compatible)
4. Hotswapping for avoiding recompilation during swapping new LoRAs 🤯
We have tested our recipe with Flux.1-Dev on both H100 and RTX 4090. We achieve at least a *2x speedup* in either of the GPUs. We believe our recipe is grounded in the reality of how LoRA-based use cases are generally served. So, we hope this will be beneficial to the community 🤗
Even though our recipe was tested primarily with NVIDIA GPUs, it should also work with AMD GPUs.
Learn the details and the full code here:
https://huggingface.co/blog/lora-fast
- 3 replies
Post
2,077
kernels 0.8.0 is out: https://github.com/huggingface/kernels/releases/tag/v0.8.0
This release refines kernel selection in the
• You can now register kernels for certain CUDA capability ranges.
• Rather than doing exact mating of modes, fall back to other compatible modes. If you are kernelizing for inference, but you only registered a training +
This release refines kernel selection in the
kernelize function:• You can now register kernels for certain CUDA capability ranges.
• Rather than doing exact mating of modes, fall back to other compatible modes. If you are kernelizing for inference, but you only registered a training +
torch.compile kernel, it will use that kernel since it is compatible with inference as well. 
- 1 reply
Post
496
You can get flash-attention 3 ⚡️ directly from the hub now using
kernels-community/flash-attn3
kernels!kernels-community/flash-attn3
Post
391
Kernels 0.7.0 is out: https://github.com/huggingface/kernels/releases/tag/v0.7.0 🚀
This release makes it possible to register multiple kernels for a layer. Do you have a super-fast kernel for inference and another kernel for training? Register them both and
This release makes it possible to register multiple kernels for a layer. Do you have a super-fast kernel for inference and another kernel for training? Register them both and
kernelize will pick the kernel depending on whether you are going to do training or inference.