docs: update activity log with HF Space deployment milestone

doc/activity.md

@@ -5124,3 +5124,179 @@ load_forecast_cols[timestamp > d1_cutoff] = np.nan
 **Next Session**: Deploy to HF Space, run time-travel validation tests
 
 ---
+
+## Session 7: HuggingFace Space Deployment (Nov 14, 2025)
+
+### Objectives
+1. Extend dataset to Oct 14, 2025 for multivariate forecasting
+2. Create production-ready Jupyter notebooks for HF Space
+3. Deploy to HuggingFace Space with Docker and GPU support
+4. Enable zero-shot inference testing on A10G GPU
+
+### 1. Dataset Extension (Oct 2025 Data Processing)
+
+**Problem**: Dataset ended Sept 30, 2025, but dynamic forecasting with `run_date=Sept 30, 23:00` requires Oct 1-14 future covariates (336 hours) for 14-day forecast.
+
+**Solution**: Process October raw data and extend unified dataset.
+
+#### Scripts Created
+- **`process_october_features.py`** (341 lines)
+  - Processes weather and ENTSO-E raw data for Oct 1-14
+  - Applies existing feature engineering modules
+  - Output: 336 rows × 840 features (weather + ENTSO-E)
+
+- **`extend_dataset.py`** (195 lines)
+  - Merges October features with 24-month baseline
+  - Handles dtype mismatches (176 columns fixed: Float64 → Int64)
+  - Adds missing JAO features (1,736 columns) filled with nulls
+  - Output: 17,880 rows × 2,553 features (Oct 2023 - Oct 14, 2025)
+
+- **`upload_to_hf.py`** (146 lines)
+  - Uploads extended dataset to HuggingFace
+  - Replaces 24-month dataset with 24.5-month version
+  - Dataset: `evgueni-p/fbmc-features-24month`
+
+**Results**:
+- Dataset extended: 17,544 → 17,880 rows (+336 hours)
+- Date range: Oct 1, 2023 - Oct 14, 2025 (24.5 months)
+- Upload verified: 17,880 rows × 2,553 columns on HuggingFace
+- No data leakage: October data only available as future covariates
+
+### 2. Production Jupyter Notebooks
+
+Created 3 notebooks for HuggingFace Space:
+
+#### **`inference_smoke_test.ipynb`**
+- **Purpose**: Quick validation (1 border × 7 days, ~1 min)
+- **Configuration**:
+  - Run date: Sept 30, 2025 23:00
+  - Forecast: Oct 1-7 (168 hours)
+  - Context: 512 hours
+  - Single border test
+- **Features**:
+  - Environment setup with GPU detection
+  - Dataset loading from HuggingFace
+  - Dynamic forecast system integration
+  - Chronos-2 model loading on GPU
+  - Zero-shot inference with visualization
+
+#### **`inference_full_14day.ipynb`**
+- **Purpose**: Production run (38 borders × 14 days, ~5 min)
+- **Configuration**:
+  - Run date: Sept 30, 2025 23:00
+  - Forecast: Oct 1-14 (336 hours)
+  - Context: 512 hours
+  - All 38 borders
+- **Features**:
+  - Batch processing with progress tracking
+  - Per-border inference timing
+  - Forecast export to parquet
+  - Sample visualizations (4 borders)
+  - Performance summary statistics
+
+#### **`evaluation.ipynb`**
+- **Purpose**: Performance analysis vs Oct 1-14 actuals
+- **Metrics**:
+  - D+1 MAE (first 24 hours) - Target: <150 MW
+  - 14-day MAE (full horizon)
+  - RMSE, MAPE across all borders
+  - Best/worst border identification
+- **Outputs**:
+  - Performance distribution histogram
+  - Forecast vs actual comparison charts
+  - CSV export of results
+
+### 3. HuggingFace Space Configuration
+
+#### **Dockerfile** (Docker SDK for GPU)
+```dockerfile
+FROM pytorch/pytorch:2.0.1-cuda11.7-cudnn8-runtime
+WORKDIR /app
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+COPY src/ ./src/
+COPY inference_smoke_test.ipynb .
+COPY inference_full_14day.ipynb .
+COPY evaluation.ipynb .
+EXPOSE 7860
+CMD ["jupyter", "lab", "--ip=0.0.0.0", "--port=7860", "--no-browser",
+     "--allow-root", "--NotebookApp.token=''", "--NotebookApp.password=''"]
+```
+
+#### **README.md** (Space Metadata)
+- SDK: `docker` (changed from `jupyterlab` - not supported)
+- Hardware: `a10g-small` (NVIDIA A10G, 24GB VRAM)
+- License: MIT
+- Features: 2,553 engineered features, 38 borders
+- Model: Amazon Chronos-2 Large (710M params)
+
+#### **requirements.txt** (GPU Dependencies)
+- Core ML: torch>=2.0.0, transformers>=4.35.0, chronos-forecasting>=1.2.0
+- Data: polars>=0.19.0, datasets>=2.14.0, pyarrow>=13.0.0
+- Viz: altair>=5.0.0
+- Jupyter: ipykernel, jupyter, jupyterlab
+
+### 4. Deployment
+
+**Git Operations**:
+```bash
+git add README.md requirements.txt Dockerfile inference_smoke_test.ipynb \
+    inference_full_14day.ipynb evaluation.ipynb src/forecasting/
+git commit -m "feat: add HF Space deployment with Docker and Jupyter notebooks"
+git push origin master  # GitHub repo
+git push hf-space master:main  # HuggingFace Space
+```
+
+**Results**:
+- HuggingFace Space: https://huggingface.co/spaces/evgueni-p/fbmc-chronos2-forecast
+- GitHub Repo: https://github.com/evgspacdmy/fbmc_chronos2
+- Dataset: https://huggingface.co/datasets/evgueni-p/fbmc-features-24month
+
+### Files Created
+- `process_october_features.py` (341 lines)
+- `extend_dataset.py` (195 lines)
+- `upload_to_hf.py` (146 lines)
+- `Dockerfile` (17 lines)
+- `inference_smoke_test.ipynb` (16 cells)
+- `inference_full_14day.ipynb` (8 cells)
+- `evaluation.ipynb` (8 cells)
+
+### Files Modified
+- `README.md` - Changed SDK from jupyterlab to docker
+- `requirements.txt` - Renamed from requirements_hf_space.txt
+
+### Key Decisions
+1. **Docker SDK**: Required for Jupyter deployment on HF Spaces (jupyterlab SDK not supported)
+2. **No Gradio**: User confirmed Jupyter notebooks only (previous Gradio app archived)
+3. **October extension**: Essential for multivariate forecasting with Sept 30 run date
+4. **JAO features**: Filled with nulls for October (no API data available)
+5. **Dataset naming**: Kept `fbmc-features-24month` (backwards compatible)
+
+### Technical Challenges Resolved
+1. **Datetime precision mismatch**: Fixed μs vs ns timezone issues in Polars
+2. **Dtype mismatches**: Cast 176 Float64 columns to Int64 to match schema
+3. **HF SDK error**: Changed from unsupported `jupyterlab` to `docker`
+4. **Missing October JAO**: Filled 1,736 columns with nulls (expected behavior)
+5. **Forward-fill**: Oct 14 ENTSO-E data missing, forward-filled from Oct 13
+
+### Testing Status
+- Dataset upload: ✅ Verified 17,880 rows on HuggingFace
+- Git deployment: ✅ Pushed to both GitHub and HF Space
+- Docker build: ⏳ Pending (Space is building)
+- GPU inference: ⏳ Pending (awaiting Space startup)
+- MAE validation: ⏳ Pending (requires running evaluation notebook)
+
+### Next Steps
+1. **Configure HF Space**: Set `HF_TOKEN` secret for private dataset access
+2. **Test smoke test**: Run on A10G GPU, verify 1-border inference works
+3. **Test full inference**: Run all 38 borders, validate 5-minute target
+4. **Run evaluation**: Compare vs Oct 1-14 actuals, document MAE
+5. **Update activity.md**: Final results and handover documentation
+
+---
+
+**Status**: [IN PROGRESS] HF Space deployed, awaiting build completion
+**Timestamp**: 2025-11-14 12:30 UTC
+**Next Session**: Configure Space secrets, test notebooks on GPU, evaluate MAE
+
+---