eyad-silx/400

HFA Parameter Mapping Debug Output

Issue

HFA model produces random predictions with extremely low probabilities (~0.000008) despite successful checkpoint loading.

Analysis Date

2025-09-15 04:26:52

Files

• hfa_parameter_debug_*.txt: Complete debug output
• hfa_parameter_analysis_*.json: Structured analysis data

Problem

The model configuration matches checkpoint dimensions (256 hidden, 8 heads, 6 layers) but parameters aren't loading correctly, resulting in random predictions instead of meaningful language modeling.

Next Steps

1. Verify exact parameter name mapping between checkpoint and model
2. Check if critical parameters (embedding, attention, lm_head) are being loaded
3. Ensure parameter values are being copied correctly (not just shapes)