evaluation.md

# Evaluation Summary for IndoHoaxDetector Space

## Metrics Overview

- **Model Architecture**: Logistic Regression trained on Indonesian news labeled as HOAX vs FAKTA.
- **Vectorizer**: TF-IDF transform created with `tfidf_vectorizer.pkl` after applying Indonesian-specific preprocessing.
- **Accuracy**: ~97.83% on the held-out validation split used during training (metadata stored in `model_metadata.txt`).
- **Precision & Recall**: Balanced on the styled labels. Precision indicates how often the model flags hoax-style text correctly; recall shows how many hoax-like examples are captured.
- **Confidence Scores**: The Gradio app exposes probability values for both labels. Use the HOAX probability as a stylistic warning, not a verdict.

## Testing Guidelines

1. Prepare a set of Indonesian news snippets (title + body) with known labels.
2. Run the preprocessing steps defined in `app.py` (lowercasing, URL stripping, non-letter removal, stop word removal, Sastrawi stemming, TF-IDF transform).
3. Use the loaded model to infer probabilities via `predict_news`.
4. Compare predictions with labels and compute metrics with any evaluation script (e.g., run the repository-level `evaluate.py` if you copy it inside this folder).

## Reporting

- Document any changes to the dataset or vectorizer.
- If you retrain the model, update this file with the new accuracy, precision, recall, and dataset description to keep the Space trustworthy.

## Caveats

- Metrics refer only to stylistic label consistency, not factual verification.
- The evaluation set may not include every possible writing style; monitor drift over time and retrain as needed.