QEVA: A Reference-Free Evaluation Metric for Narrative Video Summarization with Multimodal Question Answering
Paper β’ 2604.24052 β’ Published
AI & ML interests
The Fellowship is a network of exceptional people from different backgrounds who contribute to open-source machine learning π§ββοΈπ¦ΈββοΈπ¦Ήπ§ββοΈ
Recent Activity
woojun-jungΒ
authored a
paper 1 day ago
Post
419
Quietly baking Image β Music π΅ v3 β now running on SOTA open-source models.
π fffiloni/image-2-music-v3 | Feel free to test it and share feedback.
Just wiring together: merve/moondream3 * victor/ace-step-jam
Image β prompt β audio | Early version, will evolve | Follow: @fffiloni
π fffiloni/image-2-music-v3 | Feel free to test it and share feedback.
Just wiring together: merve/moondream3 * victor/ace-step-jam
Image β prompt β audio | Early version, will evolve | Follow: @fffiloni
prithivMLmodsΒ
posted an update 8 days ago
Post
1797
Now, a collection of various compression schemes for Qwen3.6 and the abliterated version 1 of dense models is available on the Hub. Check it out via the links below. π
π Qwen3.6-MoE: https://huggingface.co/collections/prithivMLmods/qwen36-35b-a3b-compressions
π Qwen3.6-27B Compressions: https://huggingface.co/collections/prithivMLmods/qwen36-27b-compressions
π€ > To learn more, visit the app page or the respective model pages.
π Qwen3.6-MoE: https://huggingface.co/collections/prithivMLmods/qwen36-35b-a3b-compressions
π Qwen3.6-27B Compressions: https://huggingface.co/collections/prithivMLmods/qwen36-27b-compressions
π€ > To learn more, visit the app page or the respective model pages.
Post
1786
π RB-Modulation is back on Hugging Face Spaces!
This is an older project that recently broke due to dependency changes, but itβs now fixed and running again β
π Whatβs fixed:
- GroundingDINO & LangSAM installation
- compatibility with recent environments
- GPU inference running smoothly again
π Try it here:
fffiloni/RB-Modulation
Feel free to give it a try again β feedback welcome!
This is an older project that recently broke due to dependency changes, but itβs now fixed and running again β
π Whatβs fixed:
- GroundingDINO & LangSAM installation
- compatibility with recent environments
- GPU inference running smoothly again
π Try it here:
fffiloni/RB-Modulation
Feel free to give it a try again β feedback welcome!
prithivMLmodsΒ
posted an update 13 days ago
Post
4131
HY-World-2.0 β A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds is now available on Spaces, and it works both as native Gradio components and in Gradio server mode.
> HY-World-2.0-Demo: prithivMLmods/HY-World-2.0-Demo
> HY-World-2.0 [Server Mode]: prithivMLmods/HY-World-2.0-Demo
> Featuring 3D reconstruction and Gaussian splats with the Rerun viewer, along with camera poses, depth maps, and surface normals.
> In Server Mode, Gradio is served via FastAPI, with FastAPI remaining the top-level server.
> Model: tencent/HY-World-2.0
> GitHub: https://github.com/PRITHIVSAKTHIUR/HY-World-2.0-Demo
π€To learn more, visit the app page or the respective model pages.
> HY-World-2.0-Demo: prithivMLmods/HY-World-2.0-Demo
> HY-World-2.0 [Server Mode]: prithivMLmods/HY-World-2.0-Demo
> Featuring 3D reconstruction and Gaussian splats with the Rerun viewer, along with camera poses, depth maps, and surface normals.
> In Server Mode, Gradio is served via FastAPI, with FastAPI remaining the top-level server.
> Model: tencent/HY-World-2.0
> GitHub: https://github.com/PRITHIVSAKTHIUR/HY-World-2.0-Demo
π€To learn more, visit the app page or the respective model pages.
prithivMLmodsΒ
posted an update 18 days ago
Post
6175
A new comparator on Spaces showcases Standard FLUX.2 Decoder vs. FLUX.2 Small Decoder. The Small Decoder is ~1.4Γ faster, uses ~1.4Γ less VRAM, and maintains near-identical image quality. It has ~28M parameters with narrower channels [96, 192, 384, 384] vs. [128, 256, 512, 512], and the demo supports sequence generation by running both decoders simultaneously and comparing the results side by side.
π€ Comparator: prithivMLmods/Flux.2-4B-Decoder-Comparator
π FLUX.2-small-decoder: black-forest-labs/FLUX.2-small-decoder
π GitHub: https://github.com/PRITHIVSAKTHIUR/Flux.2-4B-Encoder-Comparator
π Collection: https://huggingface.co/collections/prithivMLmods/image-generation-apps-collection
π€ > App built on the Gradio SDK. To learn more, visit the app page or the respective model pages.
π€ Comparator: prithivMLmods/Flux.2-4B-Decoder-Comparator
π FLUX.2-small-decoder: black-forest-labs/FLUX.2-small-decoder
π GitHub: https://github.com/PRITHIVSAKTHIUR/Flux.2-4B-Encoder-Comparator
π Collection: https://huggingface.co/collections/prithivMLmods/image-generation-apps-collection
π€ > App built on the Gradio SDK. To learn more, visit the app page or the respective model pages.
prithivMLmodsΒ
posted an update 19 days ago
Post
4218
Now, a collection of various compression schemes for Gemma 4 and the abliterated version 1 of dense models is available on the Hub. Check it out via the links below. π
πGemma 4 Compression(s)- https://huggingface.co/collections/prithivMLmods/gemma-4-compressions
πGemma 4 Uncensored [MAX] + Compression(s) - [`Ξ² ]- https://huggingface.co/collections/prithivMLmods/gemma-4-uncensored-max-compressions
πGemma 4 Compression(s) - MoE- https://huggingface.co/collections/prithivMLmods/gemma-4-compressions-moe
πGemma-4 F32 GGUF- https://huggingface.co/collections/prithivMLmods/gemma-4-f32-gguf
π€ > To learn more, visit the app page or the respective model pages.
πGemma 4 Compression(s)- https://huggingface.co/collections/prithivMLmods/gemma-4-compressions
πGemma 4 Uncensored [MAX] + Compression(s) - [`Ξ² ]- https://huggingface.co/collections/prithivMLmods/gemma-4-uncensored-max-compressions
πGemma 4 Compression(s) - MoE- https://huggingface.co/collections/prithivMLmods/gemma-4-compressions-moe
πGemma-4 F32 GGUF- https://huggingface.co/collections/prithivMLmods/gemma-4-f32-gguf
π€ > To learn more, visit the app page or the respective model pages.
Post
3161
β¨ PASD Magnify is back on Hugging Face Spaces
fffiloni/PASD
PASD isnβt recent, but still delivers strong results β worth restoring rather than replacing.
Getting it to run again wasnβt a simple dependency issue.
It relied on parts of diffusers that no longer exist, while moving to Gradio 6 forced a much newer HF stack β and I couldnβt modify the original source directly.
Recreating the old environment wasnβt practical.
So I patched the downloaded code at runtime before import and made it compatible with todayβs stack.
That ended up being the only approach that held without forking or freezing everything to outdated versions.
If youβve used it before (or are curious), feel free to give it another try.
fffiloni/PASD
PASD isnβt recent, but still delivers strong results β worth restoring rather than replacing.
Getting it to run again wasnβt a simple dependency issue.
It relied on parts of diffusers that no longer exist, while moving to Gradio 6 forced a much newer HF stack β and I couldnβt modify the original source directly.
Recreating the old environment wasnβt practical.
So I patched the downloaded code at runtime before import and made it compatible with todayβs stack.
That ended up being the only approach that held without forking or freezing everything to outdated versions.
If youβve used it before (or are curious), feel free to give it another try.
Post
757
π I've just published Sentence Transformers v5.4 to make the project fully multimodal for embeddings and reranking. The release also includes a modular CrossEncoder, and automatic Flash Attention 2 input flattening. Details:
You can now use SentenceTransformer and CrossEncoder with text, images, audio, and video, with the same familiar API. That means you can compute embeddings for an image and a text query using model.encode(), compare them with model.similarity(), and it just works. Models like Qwen3-VL-Embedding-2B and jinaai/jina-reranker-m0 are supported out of the box.
Beyond multimodal, I also fully modularized the CrossEncoder class. It's now a torch.nn.Sequential of composable modules, just like SentenceTransformer has been. This unlocked support for generative rerankers (CausalLM-based models like mxbai-rerank-v2 and the Qwen3 rerankers) via a new LogitScore module, which wasn't possible before without custom code.
Also, Flash Attention 2 now automatically skips padding for text-only inputs. If your batch has a mix of short and long texts, this gives you a nice speedup and lower VRAM usage for free.
I wrote a blog post walking through the multimodal features with practical examples. Check it out if you want to get started, or just point your Agent to the URL: https://huggingface.co/blog/multimodal-sentence-transformers
This release has set up the groundwork for more easily introducing late-interaction models (both text-only and multimodal) into Sentence Transformers in the next major release. I'm looking forward to it!
You can now use SentenceTransformer and CrossEncoder with text, images, audio, and video, with the same familiar API. That means you can compute embeddings for an image and a text query using model.encode(), compare them with model.similarity(), and it just works. Models like Qwen3-VL-Embedding-2B and jinaai/jina-reranker-m0 are supported out of the box.
Beyond multimodal, I also fully modularized the CrossEncoder class. It's now a torch.nn.Sequential of composable modules, just like SentenceTransformer has been. This unlocked support for generative rerankers (CausalLM-based models like mxbai-rerank-v2 and the Qwen3 rerankers) via a new LogitScore module, which wasn't possible before without custom code.
Also, Flash Attention 2 now automatically skips padding for text-only inputs. If your batch has a mix of short and long texts, this gives you a nice speedup and lower VRAM usage for free.
I wrote a blog post walking through the multimodal features with practical examples. Check it out if you want to get started, or just point your Agent to the URL: https://huggingface.co/blog/multimodal-sentence-transformers
This release has set up the groundwork for more easily introducing late-interaction models (both text-only and multimodal) into Sentence Transformers in the next major release. I'm looking forward to it!
prithivMLmodsΒ
posted an update 23 days ago
Post
2307
Now the demo for image detection based on SAM3 and Gemma-4 (*Filter) is available on Spaces, using full-fledged Transformers inference with multimodal reasoning for processed images. It also supports video segmentation (mask), video segmentation (annotation), and image click segmentation.
π€ Demo Space: prithivMLmods/SAM3-Gemma4-CUDA
π₯½ SAM3: facebook/sam3
π gemma-4-E2B-it: google/gemma-4-E2B-it
To learn more, visit the app page or the respective model pages.
π€ Demo Space: prithivMLmods/SAM3-Gemma4-CUDA
π₯½ SAM3: facebook/sam3
π gemma-4-E2B-it: google/gemma-4-E2B-it
To learn more, visit the app page or the respective model pages.
- 1 reply
prithivMLmodsΒ
posted an update 25 days ago
Post
4755
The demo for Image Detection (*Filter) based on SAM3 and Qwen-3.5 is now available on Hugging Face Spaces using Transformers inference, with multimodal reasoning for processed images, and it also supports video segmentation (mask), video segmentation (annotation), and image click segmentation.
π€ Demo Space: prithivMLmods/SAM3-Plus-Qwen3.5
π₯½ SAM3: facebook/sam3
π Qwen-3.5: Qwen/Qwen3.5-2B
To learn more, visit the app page or the respective model pages.
π€ Demo Space: prithivMLmods/SAM3-Plus-Qwen3.5
π₯½ SAM3: facebook/sam3
π Qwen-3.5: Qwen/Qwen3.5-2B
To learn more, visit the app page or the respective model pages.
- 5 replies
Post
2859
β
Back up and running!
My TIGER app is now fully working again, with fixes and full compatibility with Gradio 6 π
It lets you:
- ποΈ Separate multiple speakers from an audio file
- π¬ Extract each speaker directly from a video
- π§ Split audio into dialog, music, and sound effects (DnR)
- π₯ Apply DnR separation directly on videos
All powered by lightweight TIGER models for fast and efficient speech separation.
Try it here π fffiloni/TIGER-audio-extraction
My TIGER app is now fully working again, with fixes and full compatibility with Gradio 6 π
It lets you:
- ποΈ Separate multiple speakers from an audio file
- π¬ Extract each speaker directly from a video
- π§ Split audio into dialog, music, and sound effects (DnR)
- π₯ Apply DnR separation directly on videos
All powered by lightweight TIGER models for fast and efficient speech separation.
Try it here π fffiloni/TIGER-audio-extraction
Post
2273
AniDoc is back π
Iβve fixed the Space and brought it back to life:
- β Working again after being broken for a while
- β Updated to Gradio 6
- β Compatible with ZeroGPU
- β Output videos now preserve original resolution and FPS
I also added advanced controls so you can experiment more (tracking, seed, motion, sketch).
Try it here: fffiloni/AniDoc
Iβve fixed the Space and brought it back to life:
- β Working again after being broken for a while
- β Updated to Gradio 6
- β Compatible with ZeroGPU
- β Output videos now preserve original resolution and FPS
I also added advanced controls so you can experiment more (tracking, seed, motion, sketch).
Try it here: fffiloni/AniDoc
prithivMLmodsΒ
posted an update about 1 month ago
Post
5310
Flux-Klein-KV-Edit-Consistency demo is now available on Spaces. It preserves character identity and delivers high-quality, realistic results after edits. No need for any special prompts, just upload the image, type your prompt, and get the resulting image blazing fast.
π₯ Demo Space: prithivMLmods/flux-klein-kv-edit-consistency
π€ Model: black-forest-labs/FLUX.2-klein-9b-kv
π€ Collection: https://huggingface.co/collections/prithivMLmods/image-generation-apps-collection
π Gradio Server Mode: https://www.gradio.app/main/guides/server-mode
β Built with Headless Gradio, an alternative to using gr.Blocks for creating the frontend and triggering events, powered by FastAPI + Gradio. You can now design the frontend however you want, with continued support for APIs, MCP, and ZeroGPU.
β Gradio Server Mode is now available from gradio@v6.10.0.
To learn more, visit the app page or the respective model pages.
π₯ Demo Space: prithivMLmods/flux-klein-kv-edit-consistency
π€ Model: black-forest-labs/FLUX.2-klein-9b-kv
π€ Collection: https://huggingface.co/collections/prithivMLmods/image-generation-apps-collection
π Gradio Server Mode: https://www.gradio.app/main/guides/server-mode
β Built with Headless Gradio, an alternative to using gr.Blocks for creating the frontend and triggering events, powered by FastAPI + Gradio. You can now design the frontend however you want, with continued support for APIs, MCP, and ZeroGPU.
β Gradio Server Mode is now available from gradio@v6.10.0.
To learn more, visit the app page or the respective model pages.
prithivMLmodsΒ
posted an update about 1 month ago
Post
4479
Map-Anything v1 (Universal Feed-Forward Metric 3D Reconstruction) demo is now available on Hugging Face Spaces. Built with Gradio and integrated with Rerun, it performs multi-image and video-based 3D reconstruction, depth, normal map, and interactive measurements.
π€ Demo: prithivMLmods/Map-Anything-v1
π€ Model: facebook/map-anything-v1
π€ Hf-Papers: MapAnything: Universal Feed-Forward Metric 3D Reconstruction (2509.13414)
π€ Demo: prithivMLmods/Map-Anything-v1
π€ Model: facebook/map-anything-v1
π€ Hf-Papers: MapAnything: Universal Feed-Forward Metric 3D Reconstruction (2509.13414)
Post
4132
I brought DALLΒ·E mini back to life π€π¨
You can try it here:
fffiloni/dalle-mini-reboot
And I also built a batch version using Hugging Face Jobs (up to 50 images per prompt):
fffiloni/dalle-mini-via-jobs
The goal was to stay close to the original JAX/Flax pipeline, while integrating it with modern tooling (Gradio + Jobs).
It ended up being a fun way to revisit this model β still weird, still fun π
You can try it here:
fffiloni/dalle-mini-reboot
And I also built a batch version using Hugging Face Jobs (up to 50 images per prompt):
fffiloni/dalle-mini-via-jobs
The goal was to stay close to the original JAX/Flax pipeline, while integrating it with modern tooling (Gradio + Jobs).
It ended up being a fun way to revisit this model β still weird, still fun π
- 4 replies
prithivMLmodsΒ
posted an update about 1 month ago
Post
3134
Introducing QIE-Bbox-Studio! π₯π€
The QIE-Bbox-Studio demo is now live β more precise and packed with more options. Users can manipulate images with object removal, design addition, and even move objects from one place to another, all in just 4-step fast inference.
π€ Demo: prithivMLmods/QIE-Bbox-Studio
π GitHub: https://github.com/PRITHIVSAKTHIUR/QIE-Bbox-Studio
π Models [LoRA] :
β QIE-2511-Object-Mover-Bbox: prithivMLmods/QIE-2511-Object-Mover-Bbox
β QIE-2511-Object-Remover-Bbox-v3: prithivMLmods/QIE-2511-Object-Remover-Bbox-v3
β QIE-2511-Outfit-Design-Layout: prithivMLmods/QIE-2511-Outfit-Design-Layout
β QIE-2509-Object-Remover-Bbox-v3: prithivMLmods/QIE-2509-Object-Remover-Bbox-v3
β QIE-2509-Object-Mover-Bbox: prithivMLmods/QIE-2509-Object-Mover-Bbox
π Collection:
β Qwen Image Edit [Layout Bbox]: https://huggingface.co/collections/prithivMLmods/qwen-image-edit-layout-bbox
To learn more, visit the app page or the respective model pages.
The QIE-Bbox-Studio demo is now live β more precise and packed with more options. Users can manipulate images with object removal, design addition, and even move objects from one place to another, all in just 4-step fast inference.
π€ Demo: prithivMLmods/QIE-Bbox-Studio
π GitHub: https://github.com/PRITHIVSAKTHIUR/QIE-Bbox-Studio
π Models [LoRA] :
β QIE-2511-Object-Mover-Bbox: prithivMLmods/QIE-2511-Object-Mover-Bbox
β QIE-2511-Object-Remover-Bbox-v3: prithivMLmods/QIE-2511-Object-Remover-Bbox-v3
β QIE-2511-Outfit-Design-Layout: prithivMLmods/QIE-2511-Outfit-Design-Layout
β QIE-2509-Object-Remover-Bbox-v3: prithivMLmods/QIE-2509-Object-Remover-Bbox-v3
β QIE-2509-Object-Mover-Bbox: prithivMLmods/QIE-2509-Object-Mover-Bbox
π Collection:
β Qwen Image Edit [Layout Bbox]: https://huggingface.co/collections/prithivMLmods/qwen-image-edit-layout-bbox
To learn more, visit the app page or the respective model pages.
prithivMLmodsΒ
posted an update about 2 months ago
Post
5096
QIE-2509-Object-Remover-Bbox-v3 is a more stable version of the Qwen Image Edit visual groundingβbased object removal model. The app was previously featured in HF Spaces of the Week and is now updated with the latest Bbox-v3 LoRA adapter.
π€ Demo: prithivMLmods/QIE-Object-Remover-Bbox
π€ LoRA: prithivMLmods/QIE-2509-Object-Remover-Bbox-v3
π€ Collection: https://huggingface.co/collections/prithivMLmods/qwen-image-edit-layout-bbox
To learn more, visit the app page or the respective model pages.
π€ Demo: prithivMLmods/QIE-Object-Remover-Bbox
π€ LoRA: prithivMLmods/QIE-2509-Object-Remover-Bbox-v3
π€ Collection: https://huggingface.co/collections/prithivMLmods/qwen-image-edit-layout-bbox
To learn more, visit the app page or the respective model pages.
- 2 replies
Post
498
A clearer demo for TADA (now multilingual) ππ
I improved the public demo for TADA β a generative framework for speech modeling via textβacoustic dual alignment.
TADA models speech as a joint sequence of text tokens and acoustic tokens, using a transformer backbone to keep text and audio synchronized during generation.
The original demo already exposed these mechanisms, but the workflow made the pipeline hard to understand.
This updated demo makes the process clearer:
β’ load the model
β’ prepare a reference voice (optionally with transcript or Whisper auto-transcription)
β’ generate speech conditioned on that reference
It also adds multilingual support.
Presets are included for a few languages, but the model supports more:
English, French, Spanish, German, Arabic, Mandarin Chinese, Italian, Japanese, Polish, Portuguese
Feel free to try different voices, accents, or languages and see how the alignment behaves.
π fffiloni/tada-dual-alignment-tts-demo
Paper
TADA: A Generative Framework for Speech Modeling via Text-Acoustic Dual Alignment (2602.23068)
I improved the public demo for TADA β a generative framework for speech modeling via textβacoustic dual alignment.
TADA models speech as a joint sequence of text tokens and acoustic tokens, using a transformer backbone to keep text and audio synchronized during generation.
The original demo already exposed these mechanisms, but the workflow made the pipeline hard to understand.
This updated demo makes the process clearer:
β’ load the model
β’ prepare a reference voice (optionally with transcript or Whisper auto-transcription)
β’ generate speech conditioned on that reference
It also adds multilingual support.
Presets are included for a few languages, but the model supports more:
English, French, Spanish, German, Arabic, Mandarin Chinese, Italian, Japanese, Polish, Portuguese
Feel free to try different voices, accents, or languages and see how the alignment behaves.
π fffiloni/tada-dual-alignment-tts-demo
Paper
TADA: A Generative Framework for Speech Modeling via Text-Acoustic Dual Alignment (2602.23068)
prithivMLmodsΒ
posted an update about 2 months ago
Post
5048
The Qwen3.5 Multimodal Understanding Demo, powered by Qwen3.5-2B, is now available on HF Spaces! It is a lightweight model designed for fast image and video reasoning. Built with Gradio, the demo showcases Image QA, Video QA, object detection, and 2D point tracking, along with real-time token streaming.
π€ Demo: prithivMLmods/Qwen-3.5-HF-Demo
β Collection: https://huggingface.co/collections/prithivMLmods/multimodal-implementations
π Qwen3.5-2B: Qwen/Qwen3.5-2B
To learn more, visit the app page or the respective model pages.
π€ Demo: prithivMLmods/Qwen-3.5-HF-Demo
β Collection: https://huggingface.co/collections/prithivMLmods/multimodal-implementations
π Qwen3.5-2B: Qwen/Qwen3.5-2B
To learn more, visit the app page or the respective model pages.