---
title: Open Asr Leaderboard CL
emoji: 🥇
colorFrom: green
colorTo: indigo
sdk: gradio
app_file: app.py
pinned: true
license: apache-2.0
short_description: Open ASR Leaderboard for Chilean Spanish
sdk_version: 4.44.0
tags:
- leaderboard
---

# Chilean Spanish ASR Leaderboard

> **Simple Gradio-based leaderboard displaying ASR evaluation results for Chilean Spanish models.**

## Quick Start

This is a simplified version that displays results from a CSV file with two tabs:
- **🏅 Chilean Spanish ASR Leaderboard**: Shows model rankings based on WER and RTFx metrics
- **📝 About**: Detailed information about the evaluation methodology and datasets

### Running the Leaderboard

```bash
# Clone the repository
git clone https://github.com/aastroza/open_asr_leaderboard_cl.git
cd open_asr_leaderboard_cl

# Install dependencies
pip install gradio pandas

# Run the application
python app.py
```

The application will load results from `results.csv` and display them in a simple, clean interface.

### Results Format

The `results.csv` file should contain the following columns:
- `model_id`: The model identifier (e.g., "openai/whisper-large-v3")
- `wer`: Word Error Rate (lower is better)
- `rtfx`: Real-Time Factor (higher is better)
- Additional metadata columns (dataset, num_samples, etc.)

### Configuration

- **Title and Content**: Edit `src/about.py` to modify the title, introduction text, and about section
- **Styling**: Customize appearance in `src/display/css_html_js.py`
- **Data Processing**: Modify the `load_results()` function in `app.py` to change how results are aggregated and displayed

## About the Evaluation

This leaderboard evaluates ASR models on Chilean Spanish using three datasets:
- **Common Voice** (Chilean Spanish subset)
- **Google Chilean Spanish** 
- **Datarisas**

Models are ranked by average Word Error Rate (WER) across all datasets, with Real-Time Factor (RTFx) as a secondary metric for inference speed.

## Models Evaluated

- openai/whisper-large-v3
- openai/whisper-large-v3-turbo  
- openai/whisper-small
- rcastrovexler/whisper-small-es-cl (Chilean Spanish fine-tuned)
- nvidia/canary-1b-v2
- nvidia/parakeet-tdt-0.6b-v3
- microsoft/Phi-4-multimodal-instruct
- mistralai/Voxtral-Mini-3B-2507
- elevenlabs/scribe_v1

For detailed methodology and complete evaluation framework, see the Modal-based evaluation code in the original repository.

## Citation

```bibtex
@misc{astroza2024chilean,
  title={Chilean Spanish ASR Test Dataset},
  author={Alonso Astroza},
  year={2025},
  howpublished={\url{https://huggingface.co/datasets/astroza/es-cl-asr-test-only}}
}
```