Upload 34 files

fitness_coach/README.md

@@ -0,0 +1,102 @@
+# Fitness Coach Scoring System
+
+A comprehensive scoring system for comparing user exercise performance to reference videos using 3D pose estimation.
+
+## Overview
+
+This module provides tools to:
+- Process reference videos and generate noisy samples for scoring
+- Process user videos and extract 3D poses
+- Compare user performance to reference using noise-based scoring
+- Generate per-body-part scores and overall performance metrics
+
+## Module Structure
+
+### Core Modules (Phase 1 - COMPLETE ✓)
+
+1. **`body_parts.py`** - Body part groupings and joint metadata
+   - Defines 17-joint skeleton structure
+   - Maps joints to body part groups (arms, legs, core, etc.)
+   - Provides noise levels per joint type
+   - Exercise-specific body part focus
+
+2. **`utils.py`** - Utility functions
+   - Pose normalization and centering
+   - Joint distance calculations
+   - Sequence interpolation
+   - Spatial alignment
+
+3. **`temporal_align.py`** - Temporal alignment
+   - Dynamic Time Warping (DTW) for sequence alignment
+   - Handles sequences of different lengths
+   - Phase alignment for motion comparison
+
+4. **`noise_scoring.py`** - Noise-based scoring
+   - Generates noisy reference samples
+   - Statistical bounds calculation
+   - Per-body-part scoring
+   - Overall performance scoring
+
+## Usage Example
+
+```python
+from fitness_coach.noise_scoring import score_with_statistical_bounds
+import numpy as np
+
+# Load poses (from processed videos)
+user_poses = np.load('user_keypoints_3D.npz')['reconstruction']
+ref_poses = np.load('reference_keypoints_3D.npz')['reconstruction']
+
+# Score the user's performance
+scores = score_with_statistical_bounds(user_poses, ref_poses)
+
+print(f"Overall Score: {scores['overall_score']:.2f}")
+print(f"Body Part Scores: {scores['body_part_scores']}")
+```
+
+## Testing
+
+Run the test suite:
+```bash
+python fitness_coach/test_modules.py
+```
+
+All core modules have been tested and verified working.
+
+## Next Steps (Phase 2)
+
+1. **Reference Processor** - Process reference videos once
+2. **User Processor** - Process user videos
+3. **Comparison Module** - Full comparison pipeline
+4. **API Integration** - REST API endpoints
+
+## Dependencies
+
+- numpy
+- scipy
+- fastdtw (optional, for better temporal alignment)
+
+Install with:
+```bash
+pip install fastdtw
+```
+
+## Status
+
+✅ Phase 1: Core Infrastructure - **COMPLETE**
+- [x] Body parts module
+- [x] Utils module  
+- [x] Temporal alignment module
+- [x] Noise scoring module
+- [x] All tests passing
+
+🔄 Phase 2: Processing Pipelines - **IN PROGRESS**
+- [ ] Reference processor
+- [ ] User processor
+- [ ] Integration with existing vis.py
+
+⏳ Phase 3: Comparison & API - **PENDING**
+- [ ] Full comparison module
+- [ ] API endpoints
+- [ ] Documentation
+

fitness_coach/__init__.py

@@ -0,0 +1,7 @@
+"""
+Fitness Coach Module
+Scoring system for comparing user exercise performance to reference videos
+"""
+
+__version__ = "0.1.0"
+

fitness_coach/body_parts.py

@@ -0,0 +1,203 @@
+"""
+Body Part Groupings and Joint Metadata
+Defines how 17-joint skeleton maps to body part groups for scoring
+"""
+
+import numpy as np
+
+# Joint indices for 17-joint Human3.6M format
+# 0: Hip, 1-3: Right leg, 4-6: Left leg, 7-10: Spine/Head, 11-13: Left arm, 14-16: Right arm
+JOINT_NAMES = [
+    'Hip',              # 0
+    'RightHip',         # 1
+    'RightKnee',        # 2
+    'RightAnkle',       # 3
+    'LeftHip',          # 4
+    'LeftKnee',         # 5
+    'LeftAnkle',        # 6
+    'Spine',            # 7
+    'Thorax',           # 8
+    'Neck',             # 9
+    'Head',             # 10
+    'LeftShoulder',     # 11
+    'LeftElbow',        # 12
+    'LeftWrist',        # 13
+    'RightShoulder',    # 14
+    'RightElbow',       # 15
+    'RightWrist',       # 16
+]
+
+# Body part groupings for scoring
+JOINT_GROUPS = {
+    'right_arm': [14, 15, 16],      # Right shoulder, elbow, wrist
+    'left_arm': [11, 12, 13],       # Left shoulder, elbow, wrist
+    'right_leg': [1, 2, 3],          # Right hip, knee, ankle
+    'left_leg': [4, 5, 6],           # Left hip, knee, ankle
+    'torso': [0, 7, 8, 9, 10],      # Hip, spine, thorax, neck, head
+    'core': [0, 7, 8],              # Hip, spine, thorax (for core exercises like push-ups)
+    'upper_body': [7, 8, 9, 10, 11, 12, 13, 14, 15, 16],  # Everything above hip
+    'lower_body': [0, 1, 2, 3, 4, 5, 6],  # Everything below and including hip
+}
+
+# Noise levels per joint type (as fraction of body scale)
+# Different joints have different acceptable variation
+JOINT_NOISE_LEVELS = {
+    'core': 0.02,           # Hip, spine - very tight tolerance
+    'shoulders': 0.04,      # Shoulder joints
+    'elbows': 0.06,         # Elbow, knee
+    'wrists': 0.08,         # Wrist, ankle
+    'hands': 0.10,          # Hands, feet - most variation
+}
+
+# Map each joint to its noise level category
+JOINT_TO_NOISE_CATEGORY = {
+    0: 'core',      # Hip
+    1: 'shoulders', # Right hip (treated as shoulder-like for movement)
+    2: 'elbows',    # Right knee
+    3: 'wrists',    # Right ankle
+    4: 'shoulders', # Left hip
+    5: 'elbows',    # Left knee
+    6: 'wrists',    # Left ankle
+    7: 'core',      # Spine
+    8: 'core',      # Thorax
+    9: 'shoulders', # Neck
+    10: 'shoulders', # Head
+    11: 'shoulders', # Left shoulder
+    12: 'elbows',    # Left elbow
+    13: 'wrists',    # Left wrist
+    14: 'shoulders', # Right shoulder
+    15: 'elbows',    # Right elbow
+    16: 'wrists',    # Right wrist
+}
+
+# Joint pairs for calculating angles (parent-child relationships)
+JOINT_PAIRS = [
+    (0, 1),   # Hip -> Right Hip
+    (1, 2),   # Right Hip -> Right Knee
+    (2, 3),   # Right Knee -> Right Ankle
+    (0, 4),   # Hip -> Left Hip
+    (4, 5),   # Left Hip -> Left Knee
+    (5, 6),   # Left Knee -> Left Ankle
+    (0, 7),   # Hip -> Spine
+    (7, 8),   # Spine -> Thorax
+    (8, 9),   # Thorax -> Neck
+    (9, 10),  # Neck -> Head
+    (8, 11),  # Thorax -> Left Shoulder
+    (11, 12), # Left Shoulder -> Left Elbow
+    (12, 13), # Left Elbow -> Left Wrist
+    (8, 14),  # Thorax -> Right Shoulder
+    (14, 15), # Right Shoulder -> Right Elbow
+    (15, 16), # Right Elbow -> Right Wrist
+]
+
+
+def get_body_part_joints(part_name):
+    """
+    Get joint indices for a body part group
+    
+    Args:
+        part_name: Name of body part (e.g., 'right_arm', 'core')
+    
+    Returns:
+        List of joint indices
+    """
+    if part_name not in JOINT_GROUPS:
+        raise ValueError(f"Unknown body part: {part_name}. Available: {list(JOINT_GROUPS.keys())}")
+    return JOINT_GROUPS[part_name]
+
+
+def get_joint_noise_level(joint_idx):
+    """
+    Get noise level for a specific joint
+    
+    Args:
+        joint_idx: Joint index (0-16)
+    
+    Returns:
+        Noise level (float) as fraction of body scale
+    """
+    if joint_idx not in JOINT_TO_NOISE_CATEGORY:
+        return 0.05  # Default
+    category = JOINT_TO_NOISE_CATEGORY[joint_idx]
+    return JOINT_NOISE_LEVELS[category]
+
+
+def get_all_body_parts():
+    """
+    Get all available body part names
+    
+    Returns:
+        List of body part names
+    """
+    return list(JOINT_GROUPS.keys())
+
+
+def get_joint_name(joint_idx):
+    """
+    Get human-readable name for a joint
+    
+    Args:
+        joint_idx: Joint index (0-16)
+    
+    Returns:
+        Joint name string
+    """
+    if 0 <= joint_idx < len(JOINT_NAMES):
+        return JOINT_NAMES[joint_idx]
+    return f"Joint_{joint_idx}"
+
+
+def get_joints_for_exercise(exercise_type):
+    """
+    Get relevant body parts for a specific exercise type
+    
+    Args:
+        exercise_type: Type of exercise (e.g., 'pushup', 'squat', 'plank')
+    
+    Returns:
+        List of body part names relevant to the exercise
+    """
+    exercise_focus = {
+        'pushup': ['core', 'right_arm', 'left_arm', 'torso'],
+        'squat': ['core', 'right_leg', 'left_leg', 'torso'],
+        'plank': ['core', 'torso', 'right_arm', 'left_arm'],
+        'lunge': ['core', 'right_leg', 'left_leg', 'torso'],
+        'all': list(JOINT_GROUPS.keys()),
+    }
+    
+    return exercise_focus.get(exercise_type.lower(), exercise_focus['all'])
+
+
+def calculate_body_scale(poses):
+    """
+    Calculate body scale (hip-to-shoulder distance) for normalization
+    
+    Args:
+        poses: Array of shape [frames, 17, 3] or [17, 3]
+    
+    Returns:
+        Average body scale (float)
+    """
+    poses = np.array(poses)
+    if len(poses.shape) == 2:
+        poses = poses[np.newaxis, :, :]
+    
+    # Hip (0) to Thorax (8) distance
+    hip_to_thorax = np.linalg.norm(poses[:, 0, :] - poses[:, 8, :], axis=1)
+    return np.mean(hip_to_thorax)
+
+
+if __name__ == "__main__":
+    # Test the module
+    print("Body Part Groups:")
+    for part, joints in JOINT_GROUPS.items():
+        joint_names = [JOINT_NAMES[j] for j in joints]
+        print(f"  {part}: {joints} - {joint_names}")
+    
+    print("\nJoint Noise Levels:")
+    for i in range(17):
+        print(f"  {JOINT_NAMES[i]}: {get_joint_noise_level(i)}")
+    
+    print("\nExercise Focus (Push-up):")
+    print(f"  {get_joints_for_exercise('pushup')}")
+

fitness_coach/comparison.py

@@ -0,0 +1,487 @@
+"""
+Motion Comparison Module
+Main module that compares user poses to reference and generates scores
+"""
+
+import numpy as np
+from .temporal_align import align_poses_sequences, find_phase_alignment
+from .noise_scoring import score_with_statistical_bounds, score_with_noisy_reference
+from .utils import normalize_body_scale, center_poses, calculate_joint_distances
+from .body_parts import get_joints_for_exercise, get_body_part_joints, JOINT_GROUPS
+from .pose_invariant import pose_invariant_score_by_body_part, align_to_canonical_pose
+
+
+def compare_motions(user_poses, ref_poses, noisy_samples=None, exercise_type='pushup', 
+                    use_dtw=True, scoring_method='statistical', use_pose_invariant=True):
+    """
+    Compare user motion to reference and generate comprehensive scores
+    
+    Args:
+        user_poses: User pose sequence [frames, 17, 3]
+        ref_poses: Reference pose sequence [frames, 17, 3]
+        noisy_samples: Pre-generated noisy samples [n_samples, frames, 17, 3] (optional)
+        exercise_type: Type of exercise for body part focus
+        use_dtw: If True, use DTW for temporal alignment (slower but more accurate)
+        scoring_method: 'statistical' (faster) or 'noisy_samples' (more accurate)
+    
+    Returns:
+        Dictionary with comprehensive scoring results
+    """
+    user_poses = np.array(user_poses)
+    ref_poses = np.array(ref_poses)
+    
+    # Convert lists to arrays if needed
+    if isinstance(user_poses, list):
+        user_poses = np.array(user_poses)
+    if isinstance(ref_poses, list):
+        ref_poses = np.array(ref_poses)
+    
+    print(f"Comparing motions:")
+    print(f"  User: {len(user_poses)} frames")
+    print(f"  Reference: {len(ref_poses)} frames")
+    
+    # Step 1: Temporal alignment
+    alignment_score = None
+    if use_dtw:
+        print("\n[1/4] Aligning sequences with DTW...")
+        try:
+            user_aligned, ref_aligned, alignment_score = find_phase_alignment(user_poses, ref_poses)
+            print(f"  Alignment score: {alignment_score:.4f}")
+        except Exception as e:
+            print(f"  DTW failed, using interpolation: {e}")
+            from .utils import interpolate_sequence
+            target_length = max(len(user_poses), len(ref_poses))
+            user_aligned = interpolate_sequence(user_poses, target_length)
+            ref_aligned = interpolate_sequence(ref_poses, target_length)
+    else:
+        print("\n[1/4] Aligning sequences with interpolation...")
+        from .utils import interpolate_sequence
+        target_length = max(len(user_poses), len(ref_poses))
+        user_aligned = interpolate_sequence(user_poses, target_length)
+        ref_aligned = interpolate_sequence(ref_poses, target_length)
+    
+    # Step 2: Spatial normalization
+    print("\n[2/4] Normalizing poses...")
+    user_norm, user_scale = normalize_body_scale(user_aligned)
+    ref_norm, ref_scale = normalize_body_scale(ref_aligned, reference_scale=user_scale)
+    
+    # Center both poses at hip
+    user_centered = center_poses(user_norm)
+    ref_centered = center_poses(ref_norm)
+    
+    # Step 3: Calculate scores
+    print(f"\n[3/4] Calculating scores ({'pose-invariant' if use_pose_invariant else scoring_method} method)...")
+    
+    if use_pose_invariant:
+        # Use pose-invariant comparison (orientation-independent)
+        print("  Using pose-invariant scoring (orientation-independent)")
+        body_part_scores = pose_invariant_score_by_body_part(
+            user_aligned, 
+            ref_aligned, 
+            JOINT_GROUPS
+        )
+        
+        # Calculate overall score
+        overall_score = np.mean(list(body_part_scores.values()))
+        
+        scores = {
+            'overall_score': float(overall_score),
+            'body_part_scores': body_part_scores,
+            'frame_scores': [],
+            'per_joint_scores': [],
+            'body_part_details': {}
+        }
+    elif scoring_method == 'noisy_samples' and noisy_samples is not None:
+        # Use noisy samples method
+        # Align noisy samples too
+        from .utils import interpolate_sequence
+        target_length = len(user_centered)
+        noisy_aligned = np.array([
+            interpolate_sequence(sample, target_length)
+            for sample in noisy_samples
+        ])
+        noisy_norm = np.array([
+            normalize_body_scale(sample, reference_scale=ref_scale)[0]
+            for sample in noisy_aligned
+        ])
+        noisy_centered = np.array([center_poses(sample) for sample in noisy_norm])
+        
+        scores = score_with_noisy_reference(
+            user_centered, 
+            ref_centered, 
+            noisy_samples=noisy_centered
+        )
+    else:
+        # Use statistical bounds method (faster)
+        scores = score_with_statistical_bounds(user_centered, ref_centered)
+    
+    # Step 4: Exercise-specific analysis
+    print("\n[4/4] Generating exercise-specific feedback...")
+    relevant_parts = get_joints_for_exercise(exercise_type)
+    
+    # Filter scores to relevant body parts
+    relevant_scores = {
+        part: scores['body_part_scores'][part]
+        for part in relevant_parts
+        if part in scores['body_part_scores']
+    }
+    
+    # Calculate average for relevant parts
+    relevant_avg = np.mean(list(relevant_scores.values())) if relevant_scores else scores['overall_score']
+    
+    # Generate feedback
+    feedback = generate_feedback(scores, relevant_scores, exercise_type)
+    
+    # Compile results
+    results = {
+        'overall_score': float(scores['overall_score']),
+        'relevant_score': float(relevant_avg),  # Score for exercise-specific body parts
+        'body_part_scores': scores['body_part_scores'],
+        'relevant_body_part_scores': relevant_scores,
+        'frame_scores': scores.get('frame_scores', []),
+        'per_joint_scores': scores.get('per_joint_scores', []),
+        'feedback': feedback,
+        'exercise_type': exercise_type,
+        'num_frames_user': len(user_poses),
+        'num_frames_ref': len(ref_poses),
+        'num_frames_aligned': len(user_centered),
+        'details': {
+            'reference_poses': ref_centered,
+            'user_poses': user_poses,
+            'aligned_user_poses': user_centered,
+            'body_part_details': scores.get('body_part_details', {}),
+            'alignment_score': alignment_score if use_dtw else None,
+        }
+    }
+    
+    print(f"\n✓ Comparison complete!")
+    print(f"  Overall score: {results['overall_score']:.2f}")
+    print(f"  Relevant score: {results['relevant_score']:.2f}")
+    
+    return results
+
+
+def generate_feedback(scores, relevant_scores, exercise_type):
+    """
+    Generate human-readable feedback based on scores
+    
+    Args:
+        scores: Full scoring dictionary
+        relevant_scores: Scores for exercise-specific body parts
+        exercise_type: Type of exercise
+    
+    Returns:
+        List of feedback strings
+    """
+    feedback = []
+    
+    # Overall feedback
+    overall = scores['overall_score']
+    if overall >= 90:
+        feedback.append("Excellent form! Keep up the great work.")
+    elif overall >= 75:
+        feedback.append("Good form overall. Minor adjustments can improve your technique.")
+    elif overall >= 60:
+        feedback.append("Decent form, but there's room for improvement.")
+    else:
+        feedback.append("Focus on improving your form. Consider reviewing the reference video.")
+    
+    # Body part specific feedback
+    if exercise_type.lower() == 'pushup':
+        # Check core
+        if 'core' in relevant_scores:
+            core_score = relevant_scores['core']
+            if core_score < 70:
+                feedback.append("Keep your core engaged and back straight throughout the movement.")
+        
+        # Check arms
+        arm_scores = [relevant_scores.get('right_arm', 0), relevant_scores.get('left_arm', 0)]
+        avg_arm = np.mean(arm_scores)
+        if avg_arm < 70:
+            feedback.append("Focus on maintaining consistent arm positioning. Both arms should move symmetrically.")
+        elif abs(arm_scores[0] - arm_scores[1]) > 15:
+            feedback.append("Your arms are moving asymmetrically. Try to keep both sides balanced.")
+    
+    elif exercise_type.lower() == 'squat':
+        # Check legs
+        leg_scores = [relevant_scores.get('right_leg', 0), relevant_scores.get('left_leg', 0)]
+        avg_leg = np.mean(leg_scores)
+        if avg_leg < 70:
+            feedback.append("Focus on proper leg positioning and depth in your squats.")
+        elif abs(leg_scores[0] - leg_scores[1]) > 15:
+            feedback.append("Your legs are moving asymmetrically. Focus on balanced movement.")
+    
+    # Find worst performing body part
+    if relevant_scores:
+        worst_part = min(relevant_scores.items(), key=lambda x: x[1])
+        if worst_part[1] < 65:
+            feedback.append(f"Pay special attention to your {worst_part[0].replace('_', ' ')} - it needs the most improvement.")
+    
+    return feedback
+
+
+def score_exercise(user_video_path, reference_id='pushup', references_dir='references', 
+                   use_dtw=True, scoring_method='statistical', force_reprocess=False, use_pose_invariant=True):
+    """
+    Complete pipeline: process user video and score against reference
+    
+    Args:
+        user_video_path: Path to user video
+        reference_id: Exercise type / reference ID
+        references_dir: Directory containing references
+        use_dtw: Use DTW for alignment
+        scoring_method: Scoring method to use
+        force_reprocess: Force reprocessing even if cached data exists
+    
+    Returns:
+        Scoring results dictionary
+    """
+    from .user_processor import process_user_video
+    from .reference_processor import load_reference
+    import shutil
+    from pathlib import Path
+    
+    print("="*60)
+    print("EXERCISE SCORING PIPELINE")
+    print("="*60)
+    
+    # Load reference
+    print(f"\nLoading reference: {reference_id}")
+    ref_data = load_reference(reference_id, references_dir=references_dir)
+    ref_poses = ref_data['poses_3d']
+    noisy_samples = ref_data.get('noisy_samples')
+    metadata = ref_data['metadata']
+    
+    print(f"  Reference frames: {len(ref_poses)}")
+    print(f"  Exercise type: {metadata['exercise_type']}")
+    
+    # Clear cache if force reprocess
+    if force_reprocess:
+        cache_dir = Path('user_videos_cache') / Path(user_video_path).stem
+        if cache_dir.exists():
+            print(f"\n⚠ Clearing cache for {Path(user_video_path).name}")
+            shutil.rmtree(cache_dir)
+    
+    # Process user video (uses cache if available)
+    print(f"\nProcessing user video: {user_video_path}")
+    user_data = process_user_video(user_video_path, cleanup=False)
+    user_poses = user_data['poses_3d']
+    
+    print(f"  User frames: {len(user_poses)}")
+    
+    # Compare
+    print(f"\nComparing motions...")
+    results = compare_motions(
+        user_poses,
+        ref_poses,
+        noisy_samples=noisy_samples,
+        exercise_type=metadata['exercise_type'],
+        use_dtw=use_dtw,
+        scoring_method=scoring_method,
+        use_pose_invariant=use_pose_invariant
+    )
+    
+    return results
+
+
+if __name__ == "__main__":
+    import argparse
+    
+    parser = argparse.ArgumentParser(description='Compare user video to reference')
+    parser.add_argument('--user-video', type=str, required=True, help='Path to user video')
+    parser.add_argument('--reference', type=str, default='pushup', help='Reference ID')
+    parser.add_argument('--references-dir', type=str, default='references', help='References directory')
+    parser.add_argument('--no-dtw', action='store_true', help='Disable DTW alignment')
+    parser.add_argument('--method', type=str, default='statistical', choices=['statistical', 'noisy_samples'],
+                       help='Scoring method')
+    parser.add_argument('--force-reprocess', action='store_true', help='Force reprocessing (ignore cache)')
+    parser.add_argument('--json', action='store_true', help='Output results as JSON for API consumption')
+    parser.add_argument('--output', type=str, help='Save JSON output to file')
+    parser.add_argument('--generate-video', action='store_true', help='Generate side-by-side comparison video')
+    parser.add_argument('--video-output', type=str, help='Path for comparison video (default: comparison_<user_video>.mp4)')
+    parser.add_argument('--video-fps', type=int, default=30, help='FPS for comparison video')
+    
+    args = parser.parse_args()
+    
+    try:
+        results = score_exercise(
+            args.user_video,
+            reference_id=args.reference,
+            references_dir=args.references_dir,
+            use_dtw=not args.no_dtw,
+            scoring_method=args.method,
+            force_reprocess=args.force_reprocess
+        )
+        
+        # Format output for API/LLM consumption
+        if args.json:
+            import json
+            from pathlib import Path
+            
+            # Create clean API response
+            api_response = {
+                "status": "success",
+                "exercise": {
+                    "type": results['exercise_type'],
+                    "reference": args.reference,
+                    "user_video": str(Path(args.user_video).name)
+                },
+                "scores": {
+                    "overall": float(round(results['overall_score'], 2)),
+                    "relevant": float(round(results['relevant_score'], 2)),
+                    "body_parts": {
+                        part: float(round(score, 2)) 
+                        for part, score in results['relevant_body_part_scores'].items()
+                    }
+                },
+                "metrics": {
+                    "frames": {
+                        "user": int(results['num_frames_user']),
+                        "reference": int(results['num_frames_ref']),
+                        "aligned": int(results['num_frames_aligned'])
+                    },
+                    "alignment_quality": float(round(results['details'].get('alignment_score', 0), 4)) if results['details'].get('alignment_score') else None,
+                    "body_part_details": {
+                        part: {
+                            "position_error_avg": float(round(metrics.get('position_error', 0), 4)),
+                            "position_error_max": float(round(metrics.get('max_position_error', 0), 4)),
+                            "tolerance_threshold": float(round(metrics.get('tolerance_threshold', 0), 4)),
+                            "in_tolerance_percentage": float(round(metrics.get('in_tolerance_percentage', 0), 1))
+                        }
+                        for part, metrics in results['details'].get('body_part_details', {}).items()
+                        if part in results['relevant_body_part_scores']
+                    }
+                },
+                "feedback": results['feedback'],
+                "llm_context": {
+                    "description": f"User performed {results['exercise_type']} exercise",
+                    "scoring_method": args.method,
+                    "interpretation": {
+                        "score_range": "0-100, where 100 is perfect form matching the reference",
+                        "position_error": "Lower is better. Measures average distance from reference pose in normalized units",
+                        "in_tolerance": "Percentage of time user's form was within acceptable bounds"
+                    }
+                }
+            }
+            
+            # Output to file or stdout
+            json_output = json.dumps(api_response, indent=2)
+            if args.output:
+                with open(args.output, 'w') as f:
+                    f.write(json_output)
+                print(f"✓ Results saved to {args.output}")
+            else:
+                print(json_output)
+        else:
+            # Human-readable output
+            print("\n" + "="*60)
+            print("SCORING RESULTS")
+            print("="*60)
+            print(f"\nOverall Score: {results['overall_score']:.2f}/100")
+            print(f"Relevant Score: {results['relevant_score']:.2f}/100")
+            print(f"\nBody Part Scores:")
+            for part, score in results['relevant_body_part_scores'].items():
+                print(f"  {part.replace('_', ' ').title()}: {score:.2f}/100")
+            print(f"\nFeedback:")
+            for i, fb in enumerate(results['feedback'], 1):
+                print(f"  {i}. {fb}")
+            
+            # Debug information
+            print("\n" + "="*60)
+            print("DEBUG INFORMATION")
+            print("="*60)
+            details = results.get('details', {})
+            print(f"\nFrame Counts:")
+            print(f"  Reference frames: {len(details.get('reference_poses', []))}")
+            print(f"  User frames (original): {len(details.get('user_poses', []))}")
+            print(f"  User frames (aligned): {len(details.get('aligned_user_poses', []))}")
+            
+            if details.get('alignment_score') is not None:
+                print(f"\nAlignment:")
+                print(f"  DTW alignment score: {details['alignment_score']:.4f}")
+            
+            print(f"\nDetailed Body Part Metrics:")
+            for part, metrics in details.get('body_part_details', {}).items():
+                if part in results['relevant_body_part_scores']:
+                    print(f"\n{part.replace('_', ' ').title()}:")
+                    print(f"  Position Error (avg): {metrics.get('position_error', 0):.4f}")
+                    print(f"  Position Error (max): {metrics.get('max_position_error', 0):.4f}")
+                    print(f"  Tolerance Threshold: {metrics.get('tolerance_threshold', 0):.4f}")
+                    print(f"  In-tolerance %: {metrics.get('in_tolerance_percentage', 0):.1f}%")
+        
+        # Generate comparison video if requested
+        if args.generate_video:
+            from pathlib import Path
+            
+            print("\n" + "="*60)
+            print("GENERATING COMPARISON VIDEO")
+            print("="*60)
+            
+            try:
+                from .video_from_images import create_comparison_video_from_images
+                from .user_processor import process_user_video
+                from .reference_processor import load_reference
+                
+                # Determine output path
+                if args.video_output:
+                    video_output = args.video_output
+                else:
+                    user_video_stem = Path(args.user_video).stem
+                    video_output = f"comparison_{user_video_stem}.mp4"
+                
+                # Find the pose3D image directories
+                # User images: user_videos_cache/{video_name}/pose3D
+                user_video_name = Path(args.user_video).stem
+                user_image_dir = Path('user_videos_cache') / user_video_name / 'pose3D'
+                
+                # Reference images: references/{exercise}/temp_processing/pose3D
+                ref_data = load_reference(args.reference, references_dir=args.references_dir)
+                ref_dir = Path(ref_data['ref_dir'])
+                reference_image_dir = ref_dir / 'temp_processing' / 'pose3D'
+                
+                # Check if directories exist
+                if not user_image_dir.exists():
+                    print(f"⚠ Warning: User pose3D images not found at {user_image_dir}")
+                    print("  Attempting to process user video to generate images...")
+                    process_user_video(args.user_video, cleanup=False)
+                    user_image_dir = Path('user_videos_cache') / user_video_name / 'pose3D'
+                
+                if not reference_image_dir.exists():
+                    # Try alternative location
+                    reference_image_dir = ref_dir / 'pose3D'
+                    if not reference_image_dir.exists():
+                        raise FileNotFoundError(
+                            f"Reference pose3D images not found. Tried:\n"
+                            f"  {ref_dir / 'temp_processing' / 'pose3D'}\n"
+                            f"  {ref_dir / 'pose3D'}"
+                        )
+                
+                print(f"  User images: {user_image_dir}")
+                print(f"  Reference images: {reference_image_dir}")
+                
+                # Create the video from existing images
+                create_comparison_video_from_images(
+                    user_image_dir=str(user_image_dir),
+                    reference_image_dir=str(reference_image_dir),
+                    output_path=video_output,
+                    user_video_name="Your Form",
+                    reference_name="Correct Form",
+                    fps=args.video_fps
+                )
+                
+            except ImportError as e:
+                print(f"✗ Error: Missing dependency for video generation")
+                print(f"  {e}")
+                print("\nPlease ensure matplotlib and ffmpeg are installed:")
+                print("  pip install matplotlib")
+                print("  And install FFmpeg from: https://ffmpeg.org/download.html")
+            except Exception as e:
+                print(f"✗ Error generating comparison video: {e}")
+                import traceback
+                traceback.print_exc()
+        
+    except Exception as e:
+        print(f"\nERROR: {e}")
+        import traceback
+        traceback.print_exc()
+

fitness_coach/noise_scoring.py

@@ -0,0 +1,296 @@
+"""
+Noise-Based Scoring System
+Generates noisy reference samples and scores user poses against them
+"""
+
+import numpy as np
+from .body_parts import (
+    get_joint_noise_level,
+    calculate_body_scale,
+    JOINT_GROUPS
+)
+
+
+def create_noisy_samples(ref_poses, n_samples=100, noise_std=None, per_joint_noise=True):
+    """
+    Create noisy reference samples for scoring
+    
+    Args:
+        ref_poses: Reference poses [frames, 17, 3]
+        n_samples: Number of noisy samples to generate
+        noise_std: Overall noise standard deviation (as fraction of body scale)
+                  If None, uses per-joint noise levels
+        per_joint_noise: If True, use different noise levels per joint
+    
+    Returns:
+        noisy_samples: Array of shape [n_samples, frames, 17, 3]
+    """
+    ref_poses = np.array(ref_poses)
+    body_scale = calculate_body_scale(ref_poses)
+    
+    noisy_samples = []
+    
+    for _ in range(n_samples):
+        noisy_pose = ref_poses.copy()
+        
+        for frame_idx in range(len(ref_poses)):
+            for joint_idx in range(17):
+                if per_joint_noise:
+                    # Use joint-specific noise level
+                    joint_noise_std = get_joint_noise_level(joint_idx) * body_scale
+                else:
+                    # Use uniform noise
+                    if noise_std is None:
+                        noise_std = 0.05  # Default 5% of body scale
+                    joint_noise_std = noise_std * body_scale
+                
+                # Add Gaussian noise to each coordinate
+                noise = np.random.normal(
+                    loc=0.0,
+                    scale=joint_noise_std,
+                    size=3
+                )
+                noisy_pose[frame_idx, joint_idx, :] += noise
+        
+        noisy_samples.append(noisy_pose)
+    
+    return np.array(noisy_samples)
+
+
+def calculate_statistical_bounds(ref_poses, noise_std=0.04, confidence=0.95):
+    """
+    Calculate statistical bounds (mean ± std) for reference poses
+    
+    Args:
+        ref_poses: Reference poses [frames, 17, 3]
+        noise_std: Noise standard deviation (as fraction of body scale, default 4%)
+        confidence: Confidence level (0.95 = 95%)
+    
+    Returns:
+        mean_poses: Mean poses [frames, 17, 3]
+        lower_bound: Lower bound [frames, 17, 3]
+        upper_bound: Upper bound [frames, 17, 3]
+        tolerance: Tolerance per joint [frames, 17]
+    """
+    ref_poses = np.array(ref_poses)
+    body_scale = calculate_body_scale(ref_poses)
+    
+    # Generate many samples and calculate statistics
+    n_samples = 1000
+    noisy_samples = create_noisy_samples(
+        ref_poses, 
+        n_samples=n_samples, 
+        noise_std=noise_std,
+        per_joint_noise=False
+    )
+    
+    # Calculate mean and std
+    mean_poses = np.mean(noisy_samples, axis=0)
+    std_poses = np.std(noisy_samples, axis=0)
+    
+    # Calculate bounds based on confidence level
+    # For 95% confidence, use ~2 standard deviations
+    z_score = 1.96 if confidence == 0.95 else 2.576  # 99% confidence
+    
+    lower_bound = mean_poses - z_score * std_poses
+    upper_bound = mean_poses + z_score * std_poses
+    
+    # Tolerance is the distance from mean to bound
+    tolerance = z_score * std_poses
+    
+    return mean_poses, lower_bound, upper_bound, tolerance
+
+
+def score_with_noisy_reference(user_poses, ref_poses, noisy_samples=None, n_samples=100):
+    """
+    Score user poses against noisy reference samples
+    
+    Args:
+        user_poses: User pose sequence [frames, 17, 3]
+        ref_poses: Reference pose sequence [frames, 17, 3]
+        noisy_samples: Pre-generated noisy samples [n_samples, frames, 17, 3]
+                      If None, generates them
+        n_samples: Number of samples to generate if noisy_samples is None
+    
+    Returns:
+        scores: Dictionary with overall and per-body-part scores
+    """
+    user_poses = np.array(user_poses)
+    ref_poses = np.array(ref_poses)
+    
+    # Generate noisy samples if not provided
+    if noisy_samples is None:
+        noisy_samples = create_noisy_samples(ref_poses, n_samples=n_samples)
+    
+    # Align temporally (simple resampling for now, DTW in comparison.py)
+    from .utils import interpolate_sequence
+    target_length = max(len(user_poses), len(ref_poses))
+    user_aligned = interpolate_sequence(user_poses, target_length)
+    ref_aligned = interpolate_sequence(ref_poses, target_length)
+    noisy_aligned = np.array([
+        interpolate_sequence(sample, target_length) 
+        for sample in noisy_samples
+    ])
+    
+    # Normalize by body scale
+    from .utils import normalize_body_scale
+    user_norm, _ = normalize_body_scale(user_aligned)
+    ref_norm, ref_scale = normalize_body_scale(ref_aligned)
+    noisy_norm = np.array([
+        normalize_body_scale(sample, reference_scale=ref_scale)[0]
+        for sample in noisy_aligned
+    ])
+    
+    # Calculate scores per body part
+    body_part_scores = {}
+    frame_scores = []
+    
+    for frame_idx in range(len(user_norm)):
+        user_frame = user_norm[frame_idx]  # [17, 3]
+        ref_frame = ref_norm[frame_idx]
+        noisy_frames = noisy_norm[:, frame_idx, :, :]  # [n_samples, 17, 3]
+        
+        # Calculate distance from user to reference
+        user_to_ref_dist = np.linalg.norm(user_frame - ref_frame, axis=1)  # [17]
+        
+        # Calculate distances from each noisy sample to reference
+        noisy_to_ref_dists = np.array([
+            np.linalg.norm(noisy_frame - ref_frame, axis=1)
+            for noisy_frame in noisy_frames
+        ])  # [n_samples, 17]
+        
+        # Score: percentage of noisy samples that are "worse" than user
+        # (i.e., user is within acceptable range)
+        frame_scores_per_joint = []
+        for joint_idx in range(17):
+            user_dist = user_to_ref_dist[joint_idx]
+            noisy_dists = noisy_to_ref_dists[:, joint_idx]
+            
+            # How many noisy samples are further from reference than user?
+            better_than = np.sum(noisy_dists > user_dist)
+            score = (better_than / len(noisy_dists)) * 100
+            frame_scores_per_joint.append(score)
+        
+        frame_scores.append(frame_scores_per_joint)
+    
+    frame_scores = np.array(frame_scores)  # [frames, 17]
+    
+    # Aggregate by body part
+    for part_name, joint_indices in JOINT_GROUPS.items():
+        part_scores = frame_scores[:, joint_indices]
+        body_part_scores[part_name] = float(np.mean(part_scores))
+    
+    # Overall score
+    overall_score = float(np.mean(frame_scores))
+    
+    return {
+        'overall_score': overall_score,
+        'body_part_scores': body_part_scores,
+        'frame_scores': frame_scores.tolist(),
+        'per_joint_scores': np.mean(frame_scores, axis=0).tolist()
+    }
+
+
+def score_with_statistical_bounds(user_poses, ref_poses, noise_std=0.04):
+    """
+    Score using statistical bounds (faster than noisy samples)
+    
+    Args:
+        user_poses: User pose sequence [frames, 17, 3]
+        ref_poses: Reference pose sequence [frames, 17, 3]
+        noise_std: Noise standard deviation (as fraction of body scale)
+                   Default 0.04 = 4% tolerance, accounts for timing differences
+    
+    Returns:
+        scores: Dictionary with overall and per-body-part scores
+    """
+    user_poses = np.array(user_poses)
+    ref_poses = np.array(ref_poses)
+    
+    # Calculate bounds (already accounts for body scale)
+    mean_poses, lower_bound, upper_bound, tolerance = calculate_statistical_bounds(
+        ref_poses, noise_std=noise_std
+    )
+    
+    # Align temporally
+    from .utils import interpolate_sequence
+    target_length = max(len(user_poses), len(ref_poses))
+    user_aligned = interpolate_sequence(user_poses, target_length)
+    mean_aligned = interpolate_sequence(mean_poses, target_length)
+    
+    # Normalize poses (but not tolerance - it's already in the right scale)
+    from .utils import normalize_body_scale
+    user_norm, user_scale = normalize_body_scale(user_aligned)
+    mean_norm, _ = normalize_body_scale(mean_aligned)
+    
+    # Scale the tolerance by the same factor used for normalization
+    # This keeps it proportional to the noise_std parameter
+    body_scale = calculate_body_scale(user_aligned)
+    tolerance_scaled = tolerance * (1.0 / body_scale)
+    tolerance_aligned = interpolate_sequence(tolerance_scaled, target_length)
+    
+    # Check if user poses are within tolerance
+    distances = np.linalg.norm(user_norm - mean_norm, axis=2)  # [frames, 17]
+    tolerance_per_joint = np.linalg.norm(tolerance_aligned, axis=2)  # [frames, 17]
+    
+    # Score: percentage of time within tolerance
+    within_tolerance = distances < tolerance_per_joint
+    joint_scores = np.mean(within_tolerance, axis=0) * 100  # [17]
+    frame_scores = np.mean(within_tolerance, axis=1) * 100  # [frames]
+    
+    # Aggregate by body part with detailed metrics
+    body_part_scores = {}
+    body_part_details = {}
+    
+    for part_name, joint_indices in JOINT_GROUPS.items():
+        # Score
+        body_part_scores[part_name] = float(np.mean(joint_scores[joint_indices]))
+        
+        # Detailed metrics for this body part
+        part_distances = distances[:, joint_indices]  # [frames, num_joints_in_part]
+        part_tolerance = tolerance_per_joint[:, joint_indices]
+        part_within = within_tolerance[:, joint_indices]
+        
+        body_part_details[part_name] = {
+            'position_error': float(np.mean(part_distances)),
+            'max_position_error': float(np.max(part_distances)),
+            'in_tolerance_percentage': float(np.mean(part_within) * 100),
+            'tolerance_threshold': float(np.mean(part_tolerance)),
+        }
+    
+    overall_score = float(np.mean(frame_scores))
+    
+    return {
+        'overall_score': overall_score,
+        'body_part_scores': body_part_scores,
+        'body_part_details': body_part_details,
+        'frame_scores': frame_scores.tolist(),
+        'per_joint_scores': joint_scores.tolist()
+    }
+
+
+if __name__ == "__main__":
+    # Test noise scoring
+    print("Testing noise-based scoring...")
+    
+    # Create test data
+    ref_poses = np.random.randn(50, 17, 3)
+    user_poses = ref_poses + np.random.normal(0, 0.1, ref_poses.shape)  # Slightly different
+    
+    # Test noisy sample generation
+    noisy_samples = create_noisy_samples(ref_poses, n_samples=50)
+    print(f"Generated {len(noisy_samples)} noisy samples")
+    print(f"Noisy samples shape: {noisy_samples.shape}")
+    
+    # Test statistical bounds
+    mean, lower, upper, tolerance = calculate_statistical_bounds(ref_poses)
+    print(f"Statistical bounds calculated: mean shape {mean.shape}")
+    
+    # Test scoring
+    scores = score_with_statistical_bounds(user_poses, ref_poses)
+    print(f"\nScoring results:")
+    print(f"  Overall score: {scores['overall_score']:.2f}")
+    print(f"  Body part scores: {scores['body_part_scores']}")
+    
+    print("\nNoise scoring tests passed!")
+

fitness_coach/persona_model.py

@@ -0,0 +1,169 @@
+"""
+Fine-tuned persona model for generating coaching feedback
+
+This module provides a local alternative to the Gemini API by using
+fine-tuned transformer models for each persona.
+"""
+
+import os
+import torch
+from pathlib import Path
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+
+class PersonaModel:
+    """Fine-tuned model for generating persona-specific coaching feedback"""
+    
+    def __init__(self, persona_name, model_dir='./models'):
+        """
+        Initialize persona model
+        
+        Args:
+            persona_name: Name of persona (e.g., "Hype Beast")
+            model_dir: Directory containing fine-tuned models
+        """
+        self.persona_name = persona_name
+        persona_safe = persona_name.lower().replace(' ', '_')
+        model_path = Path(model_dir) / f'persona_{persona_safe}'
+        
+        if not model_path.exists():
+            raise FileNotFoundError(
+                f"Model not found at {model_path}. "
+                f"Please fine-tune the model first using scripts/fine_tune_persona.py"
+            )
+        
+        print(f"Loading persona model: {persona_name} from {model_path}")
+        
+        self.tokenizer = AutoTokenizer.from_pretrained(model_path)
+        self.model = AutoModelForCausalLM.from_pretrained(model_path)
+        self.model.eval()  # Set to evaluation mode
+    
+    def generate_feedback(self, objective_report, max_length=512, temperature=0.7):
+        """
+        Generate coaching feedback for given objective report
+        
+        Args:
+            objective_report: Objective performance report string
+            max_length: Maximum length of generated text
+            temperature: Sampling temperature (higher = more creative)
+        
+        Returns:
+            Generated feedback string
+        """
+        prompt = f"<|persona|>{self.persona_name}<|input|>{objective_report}<|output|>"
+        
+        inputs = self.tokenizer(
+            prompt,
+            return_tensors="pt",
+            truncation=True,
+            max_length=512
+        )
+        
+        with torch.no_grad():
+            outputs = self.model.generate(
+                **inputs,
+                max_length=max_length,
+                temperature=temperature,
+                do_sample=True,
+                pad_token_id=self.tokenizer.eos_token_id,
+                eos_token_id=self.tokenizer.convert_tokens_to_ids('<|endoftext|>') if '<|endoftext|>' in self.tokenizer.get_vocab() else self.tokenizer.eos_token_id,
+            )
+        
+        generated_text = self.tokenizer.decode(outputs[0], skip_special_tokens=False)
+        
+        # Extract output part
+        if '<|output|>' in generated_text:
+            output = generated_text.split('<|output|>')[-1]
+            if '<|endoftext|>' in output:
+                output = output.split('<|endoftext|>')[0]
+            return output.strip()
+        
+        return generated_text
+
+
+def get_persona_feedback(objective_report, persona_name, use_fine_tuned=True, fallback_to_gemini=True):
+    """
+    Get persona feedback, using fine-tuned model if available, otherwise Gemini API
+    
+    Args:
+        objective_report: Objective performance report
+        persona_name: Name of persona
+        use_fine_tuned: Try to use fine-tuned model first
+        fallback_to_gemini: Fall back to Gemini API if fine-tuned model fails
+    
+    Returns:
+        Feedback string
+    """
+    # Try fine-tuned model first
+    if use_fine_tuned:
+        try:
+            model = PersonaModel(persona_name)
+            feedback = model.generate_feedback(objective_report)
+            print(f"✅ Generated feedback using fine-tuned model for {persona_name}")
+            return feedback
+        except FileNotFoundError:
+            print(f"⚠️  Fine-tuned model not found for {persona_name}, falling back to Gemini")
+        except Exception as e:
+            print(f"⚠️  Error using fine-tuned model: {e}, falling back to Gemini")
+    
+    # Fall back to Gemini API
+    if fallback_to_gemini:
+        try:
+            import google.generativeai as genai
+            
+            api_key = os.environ.get('GEMINI_API_KEY')
+            if not api_key:
+                config_path = Path(__file__).parent.parent / 'config.txt'
+                if config_path.exists():
+                    api_key = config_path.read_text().strip()
+            
+            if not api_key:
+                raise ValueError("GEMINI_API_KEY not found")
+            
+            genai.configure(api_key=api_key)
+            
+            # Persona system instructions
+            PERSONAS = {
+                "Hype Beast": "You are The Hype Beast. Your tone is ultra-motivational, energetic, and uses enthusiastic modern slang. You focus on confidence, confidence, confidence, bringing the energy, and framing corrections as leveling up. Use many emojis and exclamation points. Make the user feel like a star. DO NOT use LaTeX formatting like $...$ for scores or percentages.",
+                "Data Scientist": "You are The Data Scientist. You speak in precise, objective terms. Your feedback uses specific metrics and quantified improvement. Translate complex biomechanical terms into clear, actionable, technical advice that a novice can understand. Use a formal, structured report format. DO NOT use LaTeX formatting like $...$ for scores or percentages.",
+                "No-Nonsense Pro": "You are The No-Nonsense Pro. You are direct, challenging, and slightly impatient with wasted effort. Your language is concise and demanding. Emphasize immediate correction and demand a higher standard of execution. Focus on 'why' the bad form wastes energy and must be fixed NOW. DO NOT use LaTeX formatting like $...$ for scores or percentages.",
+                "Mindful Aligner": "You are The Mindful Aligner. Your tone is calm, centered, and encouraging. You focus on connecting movement to breath, finding internal stability, and making gentle, internal adjustments to achieve proper alignment. Use soft, encouraging language. DO NOT use LaTeX formatting like $...$ for scores or percentages.",
+            }
+            
+            system_instruction = PERSONAS.get(persona_name, PERSONAS["Hype Beast"])
+            
+            prompt = f"""Based on the following objective performance report for a workout, you must adopt the selected persona and provide detailed, actionable coaching feedback. Start your response with a clear, concise title using a markdown H1 heading (# TITLE). DO NOT use LaTeX commands like $...$ or \\frac{{}}{{}} for scores, percentages, or numbers. Use plain text and standard Unicode symbols (like the percent sign %).
+
+**OBJECTIVE PERFORMANCE DATA:**
+---
+{objective_report}
+---"""
+            
+            # Try available models
+            model_names = [
+                "gemini-1.5-flash-latest",
+                "gemini-1.5-flash-002",
+                "gemini-1.5-pro-latest",
+                "gemini-pro",
+            ]
+            
+            for model_name in model_names:
+                try:
+                    model = genai.GenerativeModel(
+                        model_name=model_name,
+                        system_instruction=system_instruction
+                    )
+                    response = model.generate_content(prompt)
+                    print(f"✅ Generated feedback using Gemini API ({model_name})")
+                    return response.text
+                except Exception as e:
+                    continue
+            
+            raise Exception("All Gemini models failed")
+            
+        except Exception as e:
+            raise Exception(f"Failed to generate feedback: {e}")
+    
+    raise Exception("No feedback generation method available")
+
+

fitness_coach/pose_invariant.py

@@ -0,0 +1,328 @@
+"""
+Pose-invariant comparison focusing on joint angles and relative positions
+Rather than absolute 3D coordinates in space
+"""
+
+import numpy as np
+from scipy.spatial.transform import Rotation
+
+
+def align_to_canonical_pose(poses):
+    """
+    Align poses to a canonical orientation using the torso as reference
+    This removes rotation/translation variations
+    
+    Args:
+        poses: Pose sequence [frames, 17, 3]
+    
+    Returns:
+        aligned_poses: Poses in canonical orientation [frames, 17, 3]
+    """
+    poses = np.array(poses)
+    aligned_poses = np.zeros_like(poses)
+    
+    # Joint indices
+    # 0: pelvis, 1: R_hip, 2: R_knee, 3: R_ankle
+    # 4: L_hip, 5: L_knee, 6: L_ankle
+    # 7: spine, 8: thorax, 9: neck, 10: head
+    # 11: L_shoulder, 12: L_elbow, 13: L_wrist
+    # 14: R_shoulder, 15: R_elbow, 16: R_wrist
+    
+    for frame_idx in range(len(poses)):
+        pose = poses[frame_idx].copy()
+        
+        # Center at pelvis (joint 0)
+        pelvis = pose[0]
+        pose_centered = pose - pelvis
+        
+        # Define torso orientation using pelvis (0) and thorax (8)
+        pelvis_pt = pose_centered[0]  # Should be [0,0,0] now
+        thorax_pt = pose_centered[8]
+        
+        # Create coordinate system from torso
+        # Y-axis: pelvis to thorax (up direction)
+        y_axis = thorax_pt - pelvis_pt
+        y_axis = y_axis / (np.linalg.norm(y_axis) + 1e-8)
+        
+        # X-axis: perpendicular to torso in frontal plane
+        # Use shoulders to determine front direction
+        left_shoulder = pose_centered[11]
+        right_shoulder = pose_centered[14]
+        shoulder_vec = left_shoulder - right_shoulder
+        
+        # Z-axis: perpendicular to both (forward direction)
+        z_axis = np.cross(shoulder_vec, y_axis)
+        z_axis = z_axis / (np.linalg.norm(z_axis) + 1e-8)
+        
+        # Recompute X-axis to ensure orthogonality
+        x_axis = np.cross(y_axis, z_axis)
+        x_axis = x_axis / (np.linalg.norm(x_axis) + 1e-8)
+        
+        # Rotation matrix from pose frame to canonical frame
+        rotation_matrix = np.column_stack([x_axis, y_axis, z_axis])
+        
+        # Apply rotation to all joints
+        pose_aligned = pose_centered @ rotation_matrix
+        
+        aligned_poses[frame_idx] = pose_aligned
+    
+    return aligned_poses
+
+
+def compute_joint_angles(poses):
+    """
+    Compute joint angles from 3D poses
+    
+    Args:
+        poses: Pose sequence [frames, 17, 3]
+    
+    Returns:
+        angles: Joint angles [frames, n_angles]
+    """
+    poses = np.array(poses)
+    n_frames = len(poses)
+    
+    # Define bone connections for angle calculation
+    # Format: (parent_joint, child_joint_1, child_joint_2)
+    angle_triplets = [
+        # Right leg angles
+        (0, 1, 2),   # Hip-knee angle (right)
+        (1, 2, 3),   # Knee-ankle angle (right)
+        
+        # Left leg angles
+        (0, 4, 5),   # Hip-knee angle (left)
+        (4, 5, 6),   # Knee-ankle angle (left)
+        
+        # Spine angles
+        (0, 7, 8),   # Pelvis-spine-thorax
+        (7, 8, 9),   # Spine-thorax-neck
+        (8, 9, 10),  # Thorax-neck-head
+        
+        # Right arm angles
+        (8, 14, 15), # Thorax-shoulder-elbow (right)
+        (14, 15, 16),# Shoulder-elbow-wrist (right)
+        
+        # Left arm angles
+        (8, 11, 12), # Thorax-shoulder-elbow (left)
+        (11, 12, 13),# Shoulder-elbow-wrist (left)
+    ]
+    
+    angles = np.zeros((n_frames, len(angle_triplets)))
+    
+    for frame_idx in range(n_frames):
+        pose = poses[frame_idx]
+        
+        for angle_idx, (j0, j1, j2) in enumerate(angle_triplets):
+            # Vectors
+            v1 = pose[j1] - pose[j0]
+            v2 = pose[j2] - pose[j1]
+            
+            # Normalize
+            v1_norm = v1 / (np.linalg.norm(v1) + 1e-8)
+            v2_norm = v2 / (np.linalg.norm(v2) + 1e-8)
+            
+            # Angle between vectors
+            cos_angle = np.clip(np.dot(v1_norm, v2_norm), -1.0, 1.0)
+            angle = np.arccos(cos_angle)
+            
+            angles[frame_idx, angle_idx] = angle
+    
+    return angles
+
+
+def compute_relative_distances(poses):
+    """
+    Compute relative distances between key joint pairs
+    (bone lengths, normalized by body scale)
+    
+    Args:
+        poses: Pose sequence [frames, 17, 3]
+    
+    Returns:
+        distances: Relative distances [frames, n_distances]
+    """
+    poses = np.array(poses)
+    n_frames = len(poses)
+    
+    # Define important bone connections
+    bone_pairs = [
+        # Limbs
+        (1, 2),   # Right thigh
+        (2, 3),   # Right shin
+        (4, 5),   # Left thigh
+        (5, 6),   # Left shin
+        (14, 15), # Right upper arm
+        (15, 16), # Right forearm
+        (11, 12), # Left upper arm
+        (12, 13), # Left forearm
+        
+        # Torso
+        (0, 8),   # Pelvis to thorax
+        (8, 9),   # Thorax to neck
+        (9, 10),  # Neck to head
+        
+        # Shoulder width
+        (11, 14), # Left to right shoulder
+        
+        # Hip width
+        (1, 4),   # Left to right hip
+    ]
+    
+    distances = np.zeros((n_frames, len(bone_pairs)))
+    
+    for frame_idx in range(n_frames):
+        pose = poses[frame_idx]
+        
+        for dist_idx, (j1, j2) in enumerate(bone_pairs):
+            dist = np.linalg.norm(pose[j2] - pose[j1])
+            distances[frame_idx, dist_idx] = dist
+    
+    return distances
+
+
+def pose_invariant_comparison(user_poses, ref_poses):
+    """
+    Compare poses in an orientation-invariant way
+    Focuses on joint angles and relative bone lengths
+    
+    Args:
+        user_poses: User pose sequence [frames, 17, 3]
+        ref_poses: Reference pose sequence [frames, 17, 3]
+    
+    Returns:
+        angle_similarity: Similarity score based on joint angles (0-100)
+        distance_similarity: Similarity score based on bone lengths (0-100)
+        combined_score: Combined similarity score (0-100)
+    """
+    # Align both sequences to canonical orientation
+    user_aligned = align_to_canonical_pose(user_poses)
+    ref_aligned = align_to_canonical_pose(ref_poses)
+    
+    # Compute joint angles
+    user_angles = compute_joint_angles(user_aligned)
+    ref_angles = compute_joint_angles(ref_aligned)
+    
+    # Compute relative distances
+    user_distances = compute_relative_distances(user_aligned)
+    ref_distances = compute_relative_distances(ref_aligned)
+    
+    # Normalize distances by body scale
+    user_scale = np.mean(user_distances)
+    ref_scale = np.mean(ref_distances)
+    user_distances_norm = user_distances / (user_scale + 1e-8)
+    ref_distances_norm = ref_distances / (ref_scale + 1e-8)
+    
+    # Compare angles (in radians)
+    angle_diff = np.abs(user_angles - ref_angles)
+    # Convert to degrees for interpretability
+    angle_diff_deg = np.rad2deg(angle_diff)
+    
+    # Score: percentage of angles within tolerance (15 degrees)
+    angle_tolerance_deg = 15.0
+    within_angle_tolerance = angle_diff_deg < angle_tolerance_deg
+    angle_similarity = float(np.mean(within_angle_tolerance) * 100)
+    
+    # Compare relative distances
+    distance_diff = np.abs(user_distances_norm - ref_distances_norm)
+    
+    # Score: percentage of distances within tolerance (10% of normalized value)
+    distance_tolerance = 0.10
+    within_distance_tolerance = distance_diff < distance_tolerance
+    distance_similarity = float(np.mean(within_distance_tolerance) * 100)
+    
+    # Combined score (weighted average)
+    # Angles are more important for form
+    combined_score = 0.7 * angle_similarity + 0.3 * distance_similarity
+    
+    return angle_similarity, distance_similarity, combined_score
+
+
+def pose_invariant_score_by_body_part(user_poses, ref_poses, body_part_groups):
+    """
+    Compute pose-invariant scores for each body part
+    
+    Args:
+        user_poses: User pose sequence [frames, 17, 3]
+        ref_poses: Reference pose sequence [frames, 17, 3]
+        body_part_groups: Dict mapping body part names to joint indices
+    
+    Returns:
+        body_part_scores: Dict of scores per body part
+    """
+    # Align poses
+    user_aligned = align_to_canonical_pose(user_poses)
+    ref_aligned = align_to_canonical_pose(ref_poses)
+    
+    # Compute angles
+    user_angles = compute_joint_angles(user_aligned)
+    ref_angles = compute_joint_angles(ref_aligned)
+    
+    # Angle triplet to joint mapping
+    # (which joints are involved in each angle)
+    angle_to_joints = [
+        [0, 1, 2],   # Right hip-knee
+        [1, 2, 3],   # Right knee-ankle
+        [0, 4, 5],   # Left hip-knee
+        [4, 5, 6],   # Left knee-ankle
+        [0, 7, 8],   # Pelvis-spine-thorax
+        [7, 8, 9],   # Spine-thorax-neck
+        [8, 9, 10],  # Thorax-neck-head
+        [8, 14, 15], # Right thorax-shoulder-elbow
+        [14, 15, 16],# Right shoulder-elbow-wrist
+        [8, 11, 12], # Left thorax-shoulder-elbow
+        [11, 12, 13],# Left shoulder-elbow-wrist
+    ]
+    
+    body_part_scores = {}
+    
+    for part_name, joint_indices in body_part_groups.items():
+        # Find angles that involve joints from this body part
+        relevant_angle_indices = []
+        for angle_idx, angle_joints in enumerate(angle_to_joints):
+            # If any joint in the angle belongs to this body part
+            if any(j in joint_indices for j in angle_joints):
+                relevant_angle_indices.append(angle_idx)
+        
+        if relevant_angle_indices:
+            # Score based on relevant angles
+            angle_diff = np.abs(user_angles[:, relevant_angle_indices] - ref_angles[:, relevant_angle_indices])
+            angle_diff_deg = np.rad2deg(angle_diff)
+            
+            within_tolerance = angle_diff_deg < 15.0  # 15 degree tolerance
+            score = float(np.mean(within_tolerance) * 100)
+            body_part_scores[part_name] = score
+        else:
+            body_part_scores[part_name] = 100.0  # No relevant angles
+    
+    return body_part_scores
+
+
+if __name__ == "__main__":
+    # Test pose-invariant comparison
+    print("Testing pose-invariant comparison...")
+    
+    # Create test poses
+    test_poses1 = np.random.randn(100, 17, 3)
+    test_poses2 = test_poses1 + np.random.randn(100, 17, 3) * 0.1
+    
+    # Test alignment
+    aligned = align_to_canonical_pose(test_poses1)
+    print(f"Aligned poses shape: {aligned.shape}")
+    
+    # Test angles
+    angles = compute_joint_angles(test_poses1)
+    print(f"Joint angles shape: {angles.shape}")
+    
+    # Test comparison
+    angle_sim, dist_sim, combined = pose_invariant_comparison(test_poses1, test_poses2)
+    print(f"Angle similarity: {angle_sim:.2f}")
+    print(f"Distance similarity: {dist_sim:.2f}")
+    print(f"Combined score: {combined:.2f}")
+    
+    print("Tests passed!")
+
+
+
+
+
+

fitness_coach/reference_processor.py

@@ -0,0 +1,296 @@
+"""
+Reference Video Processor
+Processes reference videos once and saves noisy samples for scoring
+"""
+
+import os
+import sys
+import numpy as np
+import json
+from pathlib import Path
+
+# Add parent directory and demo directory to path
+project_root = Path(__file__).parent.parent
+sys.path.insert(0, str(project_root))
+sys.path.insert(0, str(project_root / 'demo'))
+
+from fitness_coach.noise_scoring import create_noisy_samples, calculate_statistical_bounds
+from fitness_coach.body_parts import calculate_body_scale, get_joints_for_exercise
+
+
+def process_reference_video(video_path, exercise_type='pushup', output_dir=None, n_samples=100):
+    """
+    Process a reference video and generate noisy samples for scoring
+    
+    Args:
+        video_path: Path to reference video file
+        exercise_type: Type of exercise (e.g., 'pushup', 'squat')
+        output_dir: Directory to save processed data (default: references/{exercise_type}/)
+        n_samples: Number of noisy samples to generate
+    
+    Returns:
+        Dictionary with paths to saved files and metadata
+    """
+    # Change to project root for imports to work correctly
+    original_cwd = os.getcwd()
+    os.chdir(project_root)
+    
+    try:
+        # Import after changing directory
+        from demo.vis import get_pose2D, get_pose3D
+    finally:
+        os.chdir(original_cwd)
+    
+    video_path = Path(video_path)
+    if not video_path.exists():
+        raise FileNotFoundError(f"Video not found: {video_path}")
+    
+    # Set up output directory
+    if output_dir is None:
+        output_dir = Path('references') / exercise_type
+    else:
+        output_dir = Path(output_dir)
+    
+    output_dir.mkdir(parents=True, exist_ok=True)
+    
+    print(f"Processing reference video: {video_path.name}")
+    print(f"Exercise type: {exercise_type}")
+    print(f"Output directory: {output_dir}")
+    
+    # Create temporary output directory for processing
+    temp_output = output_dir / 'temp_processing'
+    temp_output.mkdir(exist_ok=True)
+    
+    # Format output directory string (get_pose3D expects trailing slash)
+    # Use absolute path to avoid issues when changing directories
+    temp_output_abs = temp_output.resolve()
+    output_dir_str = str(temp_output_abs).replace('\\', '/')
+    if not output_dir_str.endswith('/'):
+        output_dir_str += '/'
+    
+    video_path_abs = video_path.resolve()
+    
+    # Change to project root for processing
+    os.chdir(project_root)
+    
+    # Save original argv and temporarily clear it to avoid argparse conflicts
+    original_argv = sys.argv.copy()
+    sys.argv = [sys.argv[0]]  # Keep only script name
+    
+    try:
+        # Step 1: Extract 2D poses
+        print("\n[1/4] Extracting 2D poses...")
+        try:
+            # get_pose2D expects output_dir with trailing slash
+            # It adds 'input_2D/' to it (line 95 in vis.py)
+            get_pose2D(str(video_path_abs), output_dir_str)
+        except Exception as e:
+            print(f"Error in 2D pose extraction: {e}")
+            raise
+        
+        # Step 2: Extract 3D poses
+        print("\n[2/4] Extracting 3D poses...")
+        try:
+            # get_pose3D also expects output_dir with trailing slash
+            # It looks for output_dir + 'input_2D/keypoints.npz' (line 190 in vis.py)
+            get_pose3D(str(video_path_abs), output_dir_str)
+        except Exception as e:
+            print(f"Error in 3D pose extraction: {e}")
+            raise
+    finally:
+        sys.argv = original_argv  # Restore original argv
+        os.chdir(original_cwd)
+    
+    # Step 3: Load 3D poses
+    # get_pose3D saves to output_dir + 'keypoints_3D.npz' (line 279 in vis.py)
+    keypoints_3d_path = temp_output_abs / 'keypoints_3D.npz'
+    
+    if not keypoints_3d_path.exists():
+        # Try alternative locations in case path handling differs
+        alt_paths = [
+            temp_output_abs / 'keypoints_3D.npz',
+            temp_output_abs.parent / 'keypoints_3D.npz',
+        ]
+        for alt_path in alt_paths:
+            if alt_path.exists():
+                keypoints_3d_path = alt_path
+                break
+        else:
+            # List what files actually exist to help debug
+            print(f"\nDebug: Looking for keypoints_3D.npz")
+            print(f"Expected location: {keypoints_3d_path}")
+            print(f"Files in temp_processing:")
+            if temp_output_abs.exists():
+                for item in temp_output_abs.rglob('*'):
+                    if item.is_file():
+                        print(f"  {item}")
+            raise FileNotFoundError(f"3D keypoints not found: {keypoints_3d_path}")
+    
+    keypoints_3d = np.load(str(keypoints_3d_path), allow_pickle=True)['reconstruction']
+    print(f"Loaded {len(keypoints_3d)} frames of 3D poses")
+    
+    # Convert to numpy array if needed
+    if isinstance(keypoints_3d, list):
+        keypoints_3d = np.array(keypoints_3d)
+    
+    # Step 4: Generate noisy samples
+    print(f"\n[3/4] Generating {n_samples} noisy samples...")
+    noisy_samples = create_noisy_samples(keypoints_3d, n_samples=n_samples, per_joint_noise=True)
+    print(f"Generated noisy samples shape: {noisy_samples.shape}")
+    
+    # Step 5: Calculate metadata
+    print("\n[4/4] Calculating metadata...")
+    body_scale = calculate_body_scale(keypoints_3d)
+    relevant_body_parts = get_joints_for_exercise(exercise_type)
+    
+    # Calculate statistical bounds
+    mean_poses, lower_bound, upper_bound, tolerance = calculate_statistical_bounds(
+        keypoints_3d, noise_std=0.05
+    )
+    
+    metadata = {
+        'exercise_type': exercise_type,
+        'video_path': str(video_path),
+        'video_name': video_path.stem,
+        'num_frames': len(keypoints_3d),
+        'body_scale': float(body_scale),
+        'relevant_body_parts': relevant_body_parts,
+        'n_samples': n_samples,
+        'timestamp': str(Path(video_path).stat().st_mtime) if video_path.exists() else None
+    }
+    
+    # Step 6: Save everything
+    print("\nSaving processed data...")
+    
+    # Save 3D poses
+    poses_3d_path = output_dir / 'keypoints_3D.npz'
+    np.savez_compressed(str(poses_3d_path), reconstruction=keypoints_3d)
+    print(f"  Saved 3D poses: {poses_3d_path}")
+    
+    # Save noisy samples
+    noisy_samples_path = output_dir / 'noisy_samples.npz'
+    np.savez_compressed(str(noisy_samples_path), samples=noisy_samples)
+    print(f"  Saved noisy samples: {noisy_samples_path}")
+    
+    # Save statistical bounds
+    bounds_path = output_dir / 'statistical_bounds.npz'
+    np.savez_compressed(
+        str(bounds_path),
+        mean=mean_poses,
+        lower_bound=lower_bound,
+        upper_bound=upper_bound,
+        tolerance=tolerance
+    )
+    print(f"  Saved statistical bounds: {bounds_path}")
+    
+    # Save metadata
+    metadata_path = output_dir / 'metadata.json'
+    with open(metadata_path, 'w') as f:
+        json.dump(metadata, f, indent=2)
+    print(f"  Saved metadata: {metadata_path}")
+    
+    # Clean up temporary files (optional - keep 2D poses for debugging)
+    # import shutil
+    # shutil.rmtree(temp_output, ignore_errors=True)
+    
+    print(f"\n✓ Reference video processed successfully!")
+    print(f"  Output directory: {output_dir}")
+    
+    return {
+        'output_dir': str(output_dir),
+        'poses_3d_path': str(poses_3d_path),
+        'noisy_samples_path': str(noisy_samples_path),
+        'bounds_path': str(bounds_path),
+        'metadata_path': str(metadata_path),
+        'metadata': metadata
+    }
+
+
+def load_reference(exercise_type, references_dir='references'):
+    """
+    Load a processed reference
+    
+    Args:
+        exercise_type: Type of exercise (e.g., 'pushup')
+        references_dir: Directory containing references
+    
+    Returns:
+        Dictionary with loaded data
+    """
+    ref_dir = Path(references_dir) / exercise_type
+    
+    if not ref_dir.exists():
+        raise FileNotFoundError(f"Reference not found: {ref_dir}")
+    
+    # Load metadata
+    metadata_path = ref_dir / 'metadata.json'
+    if not metadata_path.exists():
+        raise FileNotFoundError(f"Metadata not found: {metadata_path}")
+    
+    with open(metadata_path, 'r') as f:
+        metadata = json.load(f)
+    
+    # Load 3D poses
+    poses_3d_path = ref_dir / 'keypoints_3D.npz'
+    if not poses_3d_path.exists():
+        raise FileNotFoundError(f"3D poses not found: {poses_3d_path}")
+    
+    poses_3d = np.load(str(poses_3d_path), allow_pickle=True)['reconstruction']
+    if isinstance(poses_3d, list):
+        poses_3d = np.array(poses_3d)
+    
+    # Load noisy samples
+    noisy_samples_path = ref_dir / 'noisy_samples.npz'
+    noisy_samples = None
+    if noisy_samples_path.exists():
+        noisy_samples = np.load(str(noisy_samples_path), allow_pickle=True)['samples']
+    
+    # Load statistical bounds
+    bounds_path = ref_dir / 'statistical_bounds.npz'
+    bounds = None
+    if bounds_path.exists():
+        bounds_data = np.load(str(bounds_path), allow_pickle=True)
+        bounds = {
+            'mean': bounds_data['mean'],
+            'lower_bound': bounds_data['lower_bound'],
+            'upper_bound': bounds_data['upper_bound'],
+            'tolerance': bounds_data['tolerance']
+        }
+    
+    return {
+        'poses_3d': poses_3d,
+        'noisy_samples': noisy_samples,
+        'bounds': bounds,
+        'metadata': metadata,
+        'ref_dir': str(ref_dir)
+    }
+
+
+if __name__ == "__main__":
+    import argparse
+    
+    parser = argparse.ArgumentParser(description='Process reference video for scoring')
+    parser.add_argument('--video', type=str, required=True, help='Path to reference video')
+    parser.add_argument('--exercise', type=str, default='pushup', help='Exercise type')
+    parser.add_argument('--output', type=str, default=None, help='Output directory')
+    parser.add_argument('--samples', type=int, default=100, help='Number of noisy samples')
+    
+    args = parser.parse_args()
+    
+    try:
+        result = process_reference_video(
+            args.video,
+            exercise_type=args.exercise,
+            output_dir=args.output,
+            n_samples=args.samples
+        )
+        print("\n" + "="*50)
+        print("SUCCESS!")
+        print("="*50)
+        print(f"Reference saved to: {result['output_dir']}")
+    except Exception as e:
+        print(f"\nERROR: {e}")
+        import traceback
+        traceback.print_exc()
+        sys.exit(1)
+

fitness_coach/temporal_align.py

@@ -0,0 +1,174 @@
+"""
+Temporal Alignment using Dynamic Time Warping (DTW)
+Aligns sequences of different lengths for comparison
+"""
+
+import numpy as np
+try:
+    from fastdtw import fastdtw
+    from scipy.spatial.distance import euclidean
+    HAS_FASTDTW = True
+except ImportError:
+    HAS_FASTDTW = False
+    print("Warning: fastdtw not installed. Using simple interpolation instead.")
+    print("Install with: pip install fastdtw")
+
+
+def align_sequences_dtw(seq1, seq2, distance_func=None):
+    """
+    Align two sequences using Dynamic Time Warping
+    
+    Args:
+        seq1: First sequence [frames, ...]
+        seq2: Second sequence [frames, ...]
+        distance_func: Distance function (default: euclidean)
+    
+    Returns:
+        aligned_seq1, aligned_seq2: Aligned sequences of same length
+        path: DTW alignment path
+    """
+    if not HAS_FASTDTW:
+        # Fallback: simple interpolation to same length
+        target_length = max(len(seq1), len(seq2))
+        from .utils import interpolate_sequence
+        if len(seq1.shape) == 3:  # [frames, joints, coords]
+            aligned_seq1 = interpolate_sequence(seq1, target_length)
+            aligned_seq2 = interpolate_sequence(seq2, target_length)
+        else:
+            # Flatten for interpolation
+            original_shape1 = seq1.shape
+            original_shape2 = seq2.shape
+            seq1_flat = seq1.reshape(len(seq1), -1)
+            seq2_flat = seq2.reshape(len(seq2), -1)
+            aligned_seq1_flat = interpolate_sequence(seq1_flat, target_length)
+            aligned_seq2_flat = interpolate_sequence(seq2_flat, target_length)
+            aligned_seq1 = aligned_seq1_flat.reshape((target_length,) + original_shape1[1:])
+            aligned_seq2 = aligned_seq2_flat.reshape((target_length,) + original_shape2[1:])
+        return aligned_seq1, aligned_seq2, None
+    
+    # Flatten sequences for DTW
+    seq1_flat = seq1.reshape(len(seq1), -1)
+    seq2_flat = seq2.reshape(len(seq2), -1)
+    
+    # Use provided distance function or default
+    if distance_func is None:
+        distance_func = euclidean
+    
+    # Compute DTW
+    distance, path = fastdtw(seq1_flat, seq2_flat, dist=distance_func)
+    
+    # Create aligned sequences using the path
+    aligned_seq1_indices = [p[0] for p in path]
+    aligned_seq2_indices = [p[1] for p in path]
+    
+    aligned_seq1 = seq1[aligned_seq1_indices]
+    aligned_seq2 = seq2[aligned_seq2_indices]
+    
+    return aligned_seq1, aligned_seq2, path
+
+
+def align_poses_sequences(poses1, poses2):
+    """
+    Align two pose sequences temporally
+    
+    Args:
+        poses1: First pose sequence [frames, 17, 3]
+        poses2: Second pose sequence [frames, 17, 3]
+    
+    Returns:
+        aligned_poses1, aligned_poses2: Aligned pose sequences
+    """
+    poses1 = np.array(poses1)
+    poses2 = np.array(poses2)
+    
+    # Use DTW to align
+    aligned_poses1, aligned_poses2, _ = align_sequences_dtw(poses1, poses2)
+    
+    return aligned_poses1, aligned_poses2
+
+
+def find_phase_alignment(user_poses, ref_poses):
+    """
+    Find optimal phase alignment between user and reference sequences
+    Uses DTW to handle different speeds and timing
+    
+    Args:
+        user_poses: User pose sequence [frames, 17, 3]
+        ref_poses: Reference pose sequence [frames, 17, 3]
+    
+    Returns:
+        aligned_user, aligned_ref: Phase-aligned sequences
+        alignment_score: Quality of alignment (lower is better)
+    """
+    user_poses = np.array(user_poses)
+    ref_poses = np.array(ref_poses)
+    
+    # Align sequences
+    aligned_user, aligned_ref, path = align_sequences_dtw(user_poses, ref_poses)
+    
+    # Calculate alignment quality (mean distance after alignment)
+    if path is not None and HAS_FASTDTW:
+        # Calculate average distance along path
+        distances = []
+        for i, j in path:
+            dist = np.linalg.norm(user_poses[i] - ref_poses[j])
+            distances.append(dist)
+        alignment_score = np.mean(distances)
+    else:
+        # Fallback: mean distance between aligned sequences
+        alignment_score = np.mean(np.linalg.norm(aligned_user - aligned_ref, axis=2))
+    
+    return aligned_user, aligned_ref, alignment_score
+
+
+def resample_to_common_length(poses1, poses2, target_length=None):
+    """
+    Resample both sequences to common length
+    
+    Args:
+        poses1: First pose sequence [frames, 17, 3]
+        poses2: Second pose sequence [frames, 17, 3]
+        target_length: Target length (default: average of both)
+    
+    Returns:
+        resampled_poses1, resampled_poses2: Resampled sequences
+    """
+    from fitness_coach.utils import interpolate_sequence
+    
+    poses1 = np.array(poses1)
+    poses2 = np.array(poses2)
+    
+    if target_length is None:
+        target_length = (len(poses1) + len(poses2)) // 2
+    
+    resampled_poses1 = interpolate_sequence(poses1, target_length)
+    resampled_poses2 = interpolate_sequence(poses2, target_length)
+    
+    return resampled_poses1, resampled_poses2
+
+
+if __name__ == "__main__":
+    # Test temporal alignment
+    print("Testing temporal alignment...")
+    
+    # Create test sequences of different lengths
+    seq1 = np.random.randn(50, 17, 3)
+    seq2 = np.random.randn(75, 17, 3)
+    
+    print(f"Original lengths: {len(seq1)} vs {len(seq2)}")
+    
+    # Test alignment
+    aligned_seq1, aligned_seq2, path = align_sequences_dtw(seq1, seq2)
+    print(f"Aligned lengths: {len(aligned_seq1)} vs {len(aligned_seq2)}")
+    
+    if path is not None:
+        print(f"DTW path length: {len(path)}")
+    else:
+        print("Using interpolation fallback")
+    
+    # Test phase alignment
+    aligned_user, aligned_ref, score = find_phase_alignment(seq1, seq2)
+    print(f"Alignment score: {score:.4f}")
+    
+    print("Temporal alignment tests passed!")
+

fitness_coach/test_modules.py

@@ -0,0 +1,175 @@
+"""
+Test script for fitness_coach modules
+"""
+
+import numpy as np
+import sys
+import os
+
+# Add parent directory to path
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from fitness_coach.body_parts import (
+    get_body_part_joints,
+    get_joint_noise_level,
+    get_joints_for_exercise,
+    calculate_body_scale
+)
+from fitness_coach.utils import (
+    normalize_body_scale,
+    center_poses,
+    calculate_joint_distances,
+    interpolate_sequence
+)
+from fitness_coach.temporal_align import (
+    align_sequences_dtw,
+    align_poses_sequences
+)
+from fitness_coach.noise_scoring import (
+    create_noisy_samples,
+    score_with_statistical_bounds
+)
+
+
+def test_body_parts():
+    """Test body parts module"""
+    print("=" * 50)
+    print("Testing body_parts module...")
+    print("=" * 50)
+    
+    # Test getting body part joints
+    right_arm = get_body_part_joints('right_arm')
+    print(f"Right arm joints: {right_arm}")
+    assert right_arm == [14, 15, 16], "Right arm joints incorrect"
+    
+    # Test noise levels
+    hip_noise = get_joint_noise_level(0)
+    print(f"Hip noise level: {hip_noise}")
+    assert hip_noise == 0.02, "Hip noise level incorrect"
+    
+    # Test exercise focus
+    pushup_parts = get_joints_for_exercise('pushup')
+    print(f"Push-up body parts: {pushup_parts}")
+    assert 'core' in pushup_parts, "Push-up should include core"
+    
+    # Test body scale calculation
+    test_poses = np.random.randn(10, 17, 3)
+    scale = calculate_body_scale(test_poses)
+    print(f"Body scale: {scale:.4f}")
+    assert scale > 0, "Body scale should be positive"
+    
+    print("[OK] body_parts module tests passed!\n")
+
+
+def test_utils():
+    """Test utils module"""
+    print("=" * 50)
+    print("Testing utils module...")
+    print("=" * 50)
+    
+    # Test normalization
+    test_poses = np.random.randn(10, 17, 3) * 10
+    normalized, scale = normalize_body_scale(test_poses)
+    print(f"Normalization: scale = {scale:.4f}")
+    assert normalized.shape == test_poses.shape, "Normalized shape should match"
+    
+    # Test centering
+    centered = center_poses(test_poses)
+    hip_pos = centered[0, 0]
+    print(f"Centering: hip position = {hip_pos}")
+    assert np.allclose(hip_pos, [0, 0, 0]), "Hip should be at origin"
+    
+    # Test distances
+    pose1 = test_poses[0]
+    pose2 = test_poses[1]
+    dists = calculate_joint_distances(pose1, pose2)
+    print(f"Joint distances: mean = {np.mean(dists):.4f}")
+    assert len(dists) == 17, "Should have 17 joint distances"
+    
+    # Test interpolation
+    short_seq = np.random.randn(5, 17, 3)
+    long_seq = interpolate_sequence(short_seq, 10)
+    print(f"Interpolation: {len(short_seq)} -> {len(long_seq)} frames")
+    assert len(long_seq) == 10, "Interpolated length should be 10"
+    
+    print("[OK] utils module tests passed!\n")
+
+
+def test_temporal_align():
+    """Test temporal alignment module"""
+    print("=" * 50)
+    print("Testing temporal_align module...")
+    print("=" * 50)
+    
+    # Create sequences of different lengths
+    seq1 = np.random.randn(30, 17, 3)
+    seq2 = np.random.randn(50, 17, 3)
+    
+    print(f"Original lengths: {len(seq1)} vs {len(seq2)}")
+    
+    # Test alignment
+    aligned_seq1, aligned_seq2, path = align_sequences_dtw(seq1, seq2)
+    print(f"Aligned lengths: {len(aligned_seq1)} vs {len(aligned_seq2)}")
+    assert len(aligned_seq1) == len(aligned_seq2), "Aligned sequences should have same length"
+    
+    # Test pose sequence alignment
+    aligned_poses1, aligned_poses2 = align_poses_sequences(seq1, seq2)
+    print(f"Pose alignment: {len(aligned_poses1)} vs {len(aligned_poses2)}")
+    assert len(aligned_poses1) == len(aligned_poses2), "Aligned poses should have same length"
+    
+    print("[OK] temporal_align module tests passed!\n")
+
+
+def test_noise_scoring():
+    """Test noise scoring module"""
+    print("=" * 50)
+    print("Testing noise_scoring module...")
+    print("=" * 50)
+    
+    # Create test data
+    ref_poses = np.random.randn(20, 17, 3)
+    user_poses = ref_poses + np.random.normal(0, 0.05, ref_poses.shape)
+    
+    # Test noisy sample generation
+    noisy_samples = create_noisy_samples(ref_poses, n_samples=20)
+    print(f"Generated {len(noisy_samples)} noisy samples")
+    assert noisy_samples.shape == (20, 20, 17, 3), "Noisy samples shape incorrect"
+    
+    # Test scoring
+    scores = score_with_statistical_bounds(user_poses, ref_poses)
+    print(f"Overall score: {scores['overall_score']:.2f}")
+    print(f"Body part scores: {list(scores['body_part_scores'].keys())}")
+    
+    assert 'overall_score' in scores, "Should have overall_score"
+    assert 'body_part_scores' in scores, "Should have body_part_scores"
+    assert 0 <= scores['overall_score'] <= 100, "Score should be 0-100"
+    
+    print("[OK] noise_scoring module tests passed!\n")
+
+
+def main():
+    """Run all tests"""
+    print("\n" + "=" * 50)
+    print("FITNESS COACH MODULE TESTS")
+    print("=" * 50 + "\n")
+    
+    try:
+        test_body_parts()
+        test_utils()
+        test_temporal_align()
+        test_noise_scoring()
+        
+        print("=" * 50)
+        print("ALL TESTS PASSED! [OK]")
+        print("=" * 50)
+        
+    except Exception as e:
+        print(f"\n[FAILED] TEST FAILED: {e}")
+        import traceback
+        traceback.print_exc()
+        sys.exit(1)
+
+
+if __name__ == "__main__":
+    main()
+

fitness_coach/user_processor.py

@@ -0,0 +1,194 @@
+"""
+User Video Processor
+Processes user videos and extracts 3D poses for scoring
+"""
+
+import os
+import sys
+import numpy as np
+from pathlib import Path
+
+# Add parent directory and demo directory to path
+project_root = Path(__file__).parent.parent
+sys.path.insert(0, str(project_root))
+sys.path.insert(0, str(project_root / 'demo'))
+
+
+def process_user_video(video_path, output_dir=None, cleanup=True):
+    """
+    Process a user video and extract 3D poses
+    
+    Args:
+        video_path: Path to user video file
+        output_dir: Directory to save processed data (default: temp_user_processing/)
+        cleanup: If True, remove intermediate files after processing
+    
+    Returns:
+        Dictionary with paths and 3D poses
+    """
+    # Change to project root for imports to work correctly
+    original_cwd = os.getcwd()
+    os.chdir(project_root)
+    
+    try:
+        # Import after changing directory
+        from demo.vis import get_pose2D, get_pose3D
+    finally:
+        os.chdir(original_cwd)
+    
+    video_path = Path(video_path)
+    if not video_path.exists():
+        raise FileNotFoundError(f"Video not found: {video_path}")
+    
+    # Set up output directory with caching
+    if output_dir is None:
+        output_dir = Path('user_videos_cache') / video_path.stem
+    else:
+        output_dir = Path(output_dir)
+    
+    output_dir.mkdir(parents=True, exist_ok=True)
+    
+    # Check if already processed (cache hit)
+    keypoints_3d_path = output_dir / 'keypoints_3D.npz'
+    if keypoints_3d_path.exists():
+        print(f"✓ Using cached processing for: {video_path.name}")
+        print(f"  Cache location: {output_dir}")
+        keypoints_3d = np.load(str(keypoints_3d_path), allow_pickle=True)['reconstruction']
+        print(f"  Loaded {len(keypoints_3d)} frames from cache\n")
+        
+        return {
+            'keypoints_3d': keypoints_3d,
+            'poses_3d': keypoints_3d,  # Alias for compatibility
+            'video_path': video_path,
+            'output_dir': output_dir,
+            'num_frames': len(keypoints_3d)
+        }
+    
+    print(f"Processing user video: {video_path.name}")
+    print(f"Output directory: {output_dir}")
+    
+    # Format output directory string (both functions expect trailing slash)
+    # Use absolute path to avoid issues when changing directories
+    output_dir_abs = output_dir.resolve()
+    output_dir_str = str(output_dir_abs).replace('\\', '/')
+    if not output_dir_str.endswith('/'):
+        output_dir_str += '/'
+    
+    video_path_abs = video_path.resolve()
+    
+    # Change to project root for processing
+    os.chdir(project_root)
+    
+    # Save original argv and temporarily clear it to avoid argparse conflicts
+    original_argv = sys.argv.copy()
+    sys.argv = [sys.argv[0]]  # Keep only script name
+    
+    try:
+        # Step 1: Extract 2D poses
+        print("\n[1/2] Extracting 2D poses...")
+        try:
+            # get_pose2D adds 'input_2D/' to output_dir (line 95 in vis.py)
+            get_pose2D(str(video_path_abs), output_dir_str)
+        except Exception as e:
+            print(f"Error in 2D pose extraction: {e}")
+            raise
+        
+        # Step 2: Extract 3D poses
+        print("\n[2/2] Extracting 3D poses...")
+        try:
+            # get_pose3D looks for output_dir + 'input_2D/keypoints.npz' (line 190 in vis.py)
+            get_pose3D(str(video_path_abs), output_dir_str)
+        except Exception as e:
+            print(f"Error in 3D pose extraction: {e}")
+            raise
+    finally:
+        sys.argv = original_argv  # Restore original argv
+        os.chdir(original_cwd)
+    
+    # Step 3: Load 3D poses
+    # get_pose3D saves to output_dir + 'keypoints_3D.npz' (line 279 in vis.py)
+    keypoints_3d_path = output_dir_abs / 'keypoints_3D.npz'
+    if not keypoints_3d_path.exists():
+        raise FileNotFoundError(f"3D keypoints not found: {keypoints_3d_path}")
+    
+    keypoints_3d = np.load(str(keypoints_3d_path), allow_pickle=True)['reconstruction']
+    print(f"Loaded {len(keypoints_3d)} frames of 3D poses")
+    
+    # Convert to numpy array if needed
+    if isinstance(keypoints_3d, list):
+        keypoints_3d = np.array(keypoints_3d)
+    
+    result = {
+        'poses_3d': keypoints_3d,
+        'output_dir': str(output_dir),
+        'keypoints_3d_path': str(keypoints_3d_path),
+        'num_frames': len(keypoints_3d)
+    }
+    
+    # Cleanup intermediate files if requested
+    if cleanup:
+        # Keep only the 3D keypoints
+        import shutil
+        for item in output_dir.iterdir():
+            if item.is_dir() and item.name != 'input_2D':  # Keep input_2D for debugging
+                shutil.rmtree(item, ignore_errors=True)
+            elif item.is_file() and item.name != 'keypoints_3D.npz':
+                item.unlink(missing_ok=True)
+    
+    print(f"\n✓ User video processed successfully!")
+    print(f"  Frames: {len(keypoints_3d)}")
+    print(f"  Output: {output_dir}")
+    
+    return result
+
+
+def load_user_poses(keypoints_path):
+    """
+    Load user poses from a saved file
+    
+    Args:
+        keypoints_path: Path to keypoints_3D.npz file
+    
+    Returns:
+        poses_3d: Array of shape [frames, 17, 3]
+    """
+    keypoints_path = Path(keypoints_path)
+    if not keypoints_path.exists():
+        raise FileNotFoundError(f"Keypoints file not found: {keypoints_path}")
+    
+    data = np.load(str(keypoints_path), allow_pickle=True)
+    poses_3d = data['reconstruction']
+    
+    if isinstance(poses_3d, list):
+        poses_3d = np.array(poses_3d)
+    
+    return poses_3d
+
+
+if __name__ == "__main__":
+    import argparse
+    
+    parser = argparse.ArgumentParser(description='Process user video for scoring')
+    parser.add_argument('--video', type=str, required=True, help='Path to user video')
+    parser.add_argument('--output', type=str, default=None, help='Output directory')
+    parser.add_argument('--keep-files', action='store_true', help='Keep intermediate files')
+    
+    args = parser.parse_args()
+    
+    try:
+        result = process_user_video(
+            args.video,
+            output_dir=args.output,
+            cleanup=not args.keep_files
+        )
+        print("\n" + "="*50)
+        print("SUCCESS!")
+        print("="*50)
+        print(f"3D poses extracted: {result['num_frames']} frames")
+        print(f"Saved to: {result['keypoints_3d_path']}")
+    except Exception as e:
+        print(f"\nERROR: {e}")
+        import traceback
+        traceback.print_exc()
+        sys.exit(1)
+

fitness_coach/utils.py

@@ -0,0 +1,266 @@
+"""
+Utility Functions for Pose Processing
+Helper functions for normalization, distance calculations, and interpolation
+"""
+
+import numpy as np
+
+
+def normalize_body_scale(poses, reference_scale=None):
+    """
+    Normalize poses by body scale (hip-to-thorax distance)
+    
+    Args:
+        poses: Array of shape [frames, 17, 3] or [17, 3]
+        reference_scale: Optional reference scale to normalize to
+    
+    Returns:
+        Normalized poses, scale used
+    """
+    poses = np.array(poses)
+    original_shape = poses.shape
+    
+    if len(poses.shape) == 2:
+        poses = poses[np.newaxis, :, :]
+    
+    # Calculate body scale (hip to thorax distance)
+    hip_to_thorax = np.linalg.norm(poses[:, 0, :] - poses[:, 8, :], axis=1)
+    body_scale = np.mean(hip_to_thorax)
+    
+    if body_scale == 0:
+        return poses.reshape(original_shape), 1.0
+    
+    # Normalize
+    if reference_scale is not None:
+        scale_factor = reference_scale / body_scale
+    else:
+        scale_factor = 1.0 / body_scale
+    
+    normalized_poses = poses * scale_factor
+    
+    return normalized_poses.reshape(original_shape), body_scale
+
+
+def center_poses(poses, joint_idx=0):
+    """
+    Center poses at a specific joint (default: hip)
+    
+    Args:
+        poses: Array of shape [frames, 17, 3] or [17, 3]
+        joint_idx: Joint to center on (default: 0 = hip)
+    
+    Returns:
+        Centered poses
+    """
+    poses = np.array(poses)
+    original_shape = poses.shape
+    
+    if len(poses.shape) == 2:
+        poses = poses[np.newaxis, :, :]
+    
+    # Subtract the reference joint position
+    centered = poses - poses[:, joint_idx:joint_idx+1, :]
+    
+    return centered.reshape(original_shape)
+
+
+def calculate_joint_distances(pose1, pose2):
+    """
+    Calculate Euclidean distances between corresponding joints
+    
+    Args:
+        pose1: Array of shape [17, 3] or [frames, 17, 3]
+        pose2: Array of shape [17, 3] or [frames, 17, 3]
+    
+    Returns:
+        Distances per joint: [17] or [frames, 17]
+    """
+    pose1 = np.array(pose1)
+    pose2 = np.array(pose2)
+    
+    if len(pose1.shape) == 2 and len(pose2.shape) == 2:
+        # Single frame
+        distances = np.linalg.norm(pose1 - pose2, axis=1)
+    else:
+        # Multiple frames
+        if len(pose1.shape) == 2:
+            pose1 = pose1[np.newaxis, :, :]
+        if len(pose2.shape) == 2:
+            pose2 = pose2[np.newaxis, :, :]
+        
+        distances = np.linalg.norm(pose1 - pose2, axis=2)
+    
+    return distances
+
+
+def calculate_joint_angles(poses, joint_pairs):
+    """
+    Calculate angles between joint pairs
+    
+    Args:
+        poses: Array of shape [frames, 17, 3] or [17, 3]
+        joint_pairs: List of (parent, child) joint index tuples
+    
+    Returns:
+        Angles in radians: [frames, n_pairs] or [n_pairs]
+    """
+    poses = np.array(poses)
+    original_shape = poses.shape
+    
+    if len(poses.shape) == 2:
+        poses = poses[np.newaxis, :, :]
+    
+    angles = []
+    for parent_idx, child_idx in joint_pairs:
+        # Vector from parent to child
+        vectors = poses[:, child_idx, :] - poses[:, parent_idx, :]
+        
+        # Calculate angle (simplified - angle with vertical)
+        # For more accurate angles, would need to consider parent-child relationships
+        vertical = np.array([0, 1, 0])
+        vertical = np.tile(vertical, (vectors.shape[0], 1))
+        
+        # Dot product and angle
+        dot_products = np.sum(vectors * vertical, axis=1)
+        vector_norms = np.linalg.norm(vectors, axis=1)
+        vertical_norm = np.linalg.norm(vertical, axis=1)
+        
+        # Avoid division by zero
+        cosines = np.clip(dot_products / (vector_norms * vertical_norm + 1e-8), -1, 1)
+        angle = np.arccos(cosines)
+        
+        angles.append(angle)
+    
+    angles = np.array(angles).T  # [frames, n_pairs]
+    
+    if len(original_shape) == 2:
+        return angles[0]
+    return angles
+
+
+def interpolate_sequence(poses, target_length):
+    """
+    Interpolate pose sequence to target length
+    
+    Args:
+        poses: Array of shape [frames, 17, 3]
+        target_length: Target number of frames
+    
+    Returns:
+        Interpolated poses: [target_length, 17, 3]
+    """
+    poses = np.array(poses)
+    original_length = poses.shape[0]
+    
+    if original_length == target_length:
+        return poses
+    
+    # Create interpolation indices
+    original_indices = np.linspace(0, original_length - 1, original_length)
+    target_indices = np.linspace(0, original_length - 1, target_length)
+    
+    # Interpolate each joint and coordinate
+    interpolated = np.zeros((target_length, poses.shape[1], poses.shape[2]))
+    
+    for joint_idx in range(poses.shape[1]):
+        for coord_idx in range(poses.shape[2]):
+            interpolated[:, joint_idx, coord_idx] = np.interp(
+                target_indices,
+                original_indices,
+                poses[:, joint_idx, coord_idx]
+            )
+    
+    return interpolated
+
+
+def smooth_poses(poses, window_size=5):
+    """
+    Apply moving average smoothing to pose sequence
+    
+    Args:
+        poses: Array of shape [frames, 17, 3]
+        window_size: Size of smoothing window
+    
+    Returns:
+        Smoothed poses
+    """
+    poses = np.array(poses)
+    if len(poses) < window_size:
+        return poses
+    
+    # Pad with edge values
+    pad_width = window_size // 2
+    padded = np.pad(poses, ((pad_width, pad_width), (0, 0), (0, 0)), mode='edge')
+    
+    # Apply moving average
+    smoothed = np.zeros_like(poses)
+    for i in range(len(poses)):
+        smoothed[i] = np.mean(padded[i:i+window_size], axis=0)
+    
+    return smoothed
+
+
+def align_poses_spatially(poses1, poses2):
+    """
+    Align two pose sequences spatially (rotation and translation)
+    Uses Procrustes alignment
+    
+    Args:
+        poses1: Reference poses [frames, 17, 3]
+        poses2: Poses to align [frames, 17, 3]
+    
+    Returns:
+        Aligned poses2
+    """
+    from scipy.spatial.transform import Rotation
+    
+    poses1 = np.array(poses1)
+    poses2 = np.array(poses2)
+    
+    # Center both
+    poses1_centered = center_poses(poses1)
+    poses2_centered = center_poses(poses2)
+    
+    # For each frame, find optimal rotation
+    aligned = np.zeros_like(poses2_centered)
+    
+    for frame_idx in range(len(poses1_centered)):
+        p1 = poses1_centered[frame_idx]
+        p2 = poses2_centered[frame_idx]
+        
+        # Find rotation using SVD (Procrustes)
+        H = p2.T @ p1
+        U, S, Vt = np.linalg.svd(H)
+        R = Vt.T @ U.T
+        
+        # Apply rotation
+        aligned[frame_idx] = p2 @ R.T
+    
+    return aligned
+
+
+if __name__ == "__main__":
+    # Test functions
+    print("Testing utility functions...")
+    
+    # Create dummy pose data
+    test_poses = np.random.randn(10, 17, 3)
+    
+    # Test normalization
+    normalized, scale = normalize_body_scale(test_poses)
+    print(f"Normalization: original scale ~{scale:.2f}")
+    
+    # Test centering
+    centered = center_poses(test_poses)
+    print(f"Centering: hip position = {centered[0, 0]}")
+    
+    # Test distances
+    dists = calculate_joint_distances(test_poses[0], test_poses[1])
+    print(f"Joint distances: mean = {np.mean(dists):.2f}")
+    
+    # Test interpolation
+    interpolated = interpolate_sequence(test_poses, 20)
+    print(f"Interpolation: {test_poses.shape[0]} -> {interpolated.shape[0]} frames")
+    
+    print("All tests passed!")
+

fitness_coach/video_comparison.py

@@ -0,0 +1,196 @@
+"""
+Generate side-by-side comparison videos of user vs reference 3D poses.
+Uses the same visualization as the original pose3D images.
+"""
+import numpy as np
+import matplotlib.pyplot as plt
+from matplotlib.animation import FuncAnimation, FFMpegWriter
+import matplotlib
+matplotlib.use('Agg')  # Use non-interactive backend
+from pathlib import Path
+import argparse
+import sys
+import os
+
+# Import the original show3Dpose function from demo/vis.py
+# Add demo directory to path
+project_root = Path(__file__).parent.parent
+demo_path = str(project_root / 'demo')
+if demo_path not in sys.path:
+    sys.path.insert(0, demo_path)
+
+from vis import show3Dpose
+
+
+def load_3d_poses(pose_file):
+    """Load 3D poses from npz file."""
+    data = np.load(pose_file, allow_pickle=True)
+    if 'reconstruction' in data:
+        poses = data['reconstruction']
+    elif 'poses_3d' in data:
+        poses = data['poses_3d']
+    else:
+        # Try to get the first array
+        poses = data[list(data.keys())[0]]
+    
+    return poses
+
+
+def plot_pose_3d(ax, pose, title):
+    """Plot a single 3D pose using the original show3Dpose function."""
+    ax.clear()
+    
+    # Use the original show3Dpose function (same as pose3D images)
+    show3Dpose(pose, ax)
+    
+    # Add title
+    ax.set_title(title, fontsize=12, fontweight='bold', pad=10)
+
+
+def create_comparison_video(user_poses, reference_poses, output_path, 
+                            user_video_name="User", reference_name="Reference",
+                            fps=30, elev=15, azim=70):
+    """
+    Create a side-by-side comparison video.
+    
+    Args:
+        user_poses: User 3D poses (N_frames, 17, 3)
+        reference_poses: Reference 3D poses (N_frames, 17, 3)
+        output_path: Path to save output video
+        user_video_name: Display name for user
+        reference_name: Display name for reference
+        fps: Frames per second for output video
+        elev: Elevation angle for 3D view
+        azim: Azimuth angle for 3D view
+    """
+    print(f"\nCreating comparison video...")
+    print(f"  User frames: {len(user_poses)}")
+    print(f"  Reference frames: {len(reference_poses)}")
+    
+    # Ensure same number of frames (use minimum)
+    n_frames = min(len(user_poses), len(reference_poses))
+    user_poses = user_poses[:n_frames]
+    reference_poses = reference_poses[:n_frames]
+    
+    # Create figure with two subplots
+    fig = plt.figure(figsize=(16, 8))
+    ax1 = fig.add_subplot(121, projection='3d')
+    ax2 = fig.add_subplot(122, projection='3d')
+    
+    # Add main title
+    fig.suptitle('Exercise Form Comparison', fontsize=16, fontweight='bold')
+    
+    def update(frame):
+        """Update function for animation."""
+        plot_pose_3d(ax1, reference_poses[frame], 
+                    f'{reference_name}\nFrame {frame+1}/{n_frames}')
+        plot_pose_3d(ax2, user_poses[frame], 
+                    f'{user_video_name}\nFrame {frame+1}/{n_frames}')
+        
+        if frame % 30 == 0:
+            print(f"  Progress: {frame}/{n_frames} frames ({100*frame//n_frames}%)")
+        
+        return ax1, ax2
+    
+    # Create animation
+    anim = FuncAnimation(fig, update, frames=n_frames, 
+                        interval=1000/fps, blit=False)
+    
+    # Save video - try MP4 first, fall back to GIF if FFmpeg not available
+    print(f"  Saving video to: {output_path}")
+    
+    # Try MP4 first (requires FFmpeg)
+    try:
+        writer = FFMpegWriter(fps=fps, bitrate=5000, codec='libx264')
+        anim.save(str(output_path), writer=writer, dpi=100)
+        print(f"✓ Video saved successfully!")
+        print(f"  Output: {output_path}")
+        print(f"  Duration: {n_frames/fps:.2f} seconds")
+        print(f"  Format: MP4")
+    except (FileNotFoundError, OSError) as e:
+        # FFmpeg not found, try GIF instead
+        print(f"  ⚠ FFmpeg not found, saving as GIF instead...")
+        gif_path = str(output_path).replace('.mp4', '.gif')
+        
+        try:
+            # Use Pillow writer for GIF (built into matplotlib)
+            from matplotlib.animation import PillowWriter
+            writer = PillowWriter(fps=fps)
+            anim.save(gif_path, writer=writer, dpi=100)
+            print(f"✓ GIF saved successfully!")
+            print(f"  Output: {gif_path}")
+            print(f"  Duration: {n_frames/fps:.2f} seconds")
+            print(f"  Format: GIF")
+            print(f"\n  Note: For MP4 format, install FFmpeg:")
+            print(f"    conda install -c conda-forge ffmpeg")
+            print(f"    or: winget install ffmpeg")
+        except Exception as gif_error:
+            print(f"✗ Error saving GIF: {gif_error}")
+            print(f"\nOriginal MP4 error: {e}")
+            print("\nTo enable MP4 output, install FFmpeg:")
+            print("  conda install -c conda-forge ffmpeg")
+            print("  or: winget install ffmpeg")
+            raise
+    except Exception as e:
+        print(f"✗ Error saving video: {e}")
+        raise
+    finally:
+        plt.close(fig)
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description='Generate side-by-side comparison video of 3D poses'
+    )
+    parser.add_argument('--user-poses', required=True,
+                       help='Path to user 3D poses npz file')
+    parser.add_argument('--reference-poses', required=True,
+                       help='Path to reference 3D poses npz file')
+    parser.add_argument('--output', default='comparison_output.mp4',
+                       help='Output video path')
+    parser.add_argument('--user-name', default='Your Form',
+                       help='Display name for user')
+    parser.add_argument('--reference-name', default='Correct Form',
+                       help='Display name for reference')
+    parser.add_argument('--fps', type=int, default=30,
+                       help='Frames per second')
+    parser.add_argument('--elev', type=float, default=15,
+                       help='Elevation angle for 3D view (degrees)')
+    parser.add_argument('--azim', type=float, default=70,
+                       help='Azimuth angle for 3D view (degrees)')
+    
+    args = parser.parse_args()
+    
+    print("="*60)
+    print("3D POSE COMPARISON VIDEO GENERATOR")
+    print("="*60)
+    
+    # Load poses
+    print(f"\nLoading user poses from: {args.user_poses}")
+    user_poses = load_3d_poses(args.user_poses)
+    print(f"  Loaded {len(user_poses)} frames")
+    
+    print(f"\nLoading reference poses from: {args.reference_poses}")
+    reference_poses = load_3d_poses(args.reference_poses)
+    print(f"  Loaded {len(reference_poses)} frames")
+    
+    # Create video
+    create_comparison_video(
+        user_poses=user_poses,
+        reference_poses=reference_poses,
+        output_path=args.output,
+        user_video_name=args.user_name,
+        reference_name=args.reference_name,
+        fps=args.fps,
+        elev=args.elev,
+        azim=args.azim
+    )
+    
+    print("\n" + "="*60)
+    print("Done!")
+    print("="*60)
+
+
+if __name__ == '__main__':
+    main()
+

fitness_coach/video_from_images.py

@@ -0,0 +1,181 @@
+"""
+Generate side-by-side comparison videos from existing pose3D images.
+Much simpler - just combines the existing PNG images!
+"""
+import numpy as np
+from PIL import Image
+import glob
+from pathlib import Path
+import argparse
+from matplotlib.animation import FuncAnimation, FFMpegWriter, PillowWriter
+import matplotlib.pyplot as plt
+import matplotlib
+matplotlib.use('Agg')
+
+
+def load_image_sequence(image_dir):
+    """Load all PNG images from a directory, sorted by filename."""
+    image_dir = Path(image_dir)
+    if not image_dir.exists():
+        raise FileNotFoundError(f"Image directory not found: {image_dir}")
+    
+    # Find all PNG files matching the pattern (e.g., 0000_3D.png, 0001_3D.png)
+    image_files = sorted(glob.glob(str(image_dir / '*_3D.png')))
+    
+    if not image_files:
+        raise FileNotFoundError(f"No pose3D images found in {image_dir}")
+    
+    print(f"  Found {len(image_files)} images in {image_dir}")
+    return image_files
+
+
+def create_comparison_video_from_images(user_image_dir, reference_image_dir, output_path,
+                                       user_video_name="Your Form", reference_name="Correct Form",
+                                       fps=30):
+    """
+    Create side-by-side video from existing pose3D images.
+    
+    Args:
+        user_image_dir: Directory containing user pose3D images
+        reference_image_dir: Directory containing reference pose3D images
+        output_path: Path to save output video
+        user_video_name: Display name for user
+        reference_name: Display name for reference
+        fps: Frames per second
+    """
+    print(f"\nCreating comparison video from existing images...")
+    
+    # Load image sequences
+    print(f"\nLoading user images from: {user_image_dir}")
+    user_images = load_image_sequence(user_image_dir)
+    
+    print(f"\nLoading reference images from: {reference_image_dir}")
+    reference_images = load_image_sequence(reference_image_dir)
+    
+    # Use minimum length to ensure both sequences are the same
+    n_frames = min(len(user_images), len(reference_images))
+    user_images = user_images[:n_frames]
+    reference_images = reference_images[:n_frames]
+    
+    print(f"\n  Using {n_frames} frames for comparison")
+    
+    # Load first images to get dimensions
+    user_img = Image.open(user_images[0])
+    ref_img = Image.open(reference_images[0])
+    
+    # Get dimensions (assuming they're similar)
+    img_height = max(user_img.height, ref_img.height)
+    img_width = max(user_img.width, ref_img.width)
+    
+    # Create figure for side-by-side display
+    fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(16, 8))
+    ax1.axis('off')
+    ax2.axis('off')
+    
+    # Add titles
+    fig.suptitle('Exercise Form Comparison', fontsize=16, fontweight='bold')
+    ax1.set_title(f'{reference_name}', fontsize=14, fontweight='bold', pad=10)
+    ax2.set_title(f'{user_video_name}', fontsize=14, fontweight='bold', pad=10)
+    
+    def update(frame):
+        """Update function for animation."""
+        # Load images
+        ref_img = Image.open(reference_images[frame])
+        user_img = Image.open(user_images[frame])
+        
+        # Display images
+        ax1.clear()
+        ax1.imshow(ref_img)
+        ax1.axis('off')
+        ax1.set_title(f'{reference_name}\nFrame {frame+1}/{n_frames}', 
+                     fontsize=12, fontweight='bold', pad=10)
+        
+        ax2.clear()
+        ax2.imshow(user_img)
+        ax2.axis('off')
+        ax2.set_title(f'{user_video_name}\nFrame {frame+1}/{n_frames}', 
+                     fontsize=12, fontweight='bold', pad=10)
+        
+        if frame % 30 == 0:
+            print(f"  Progress: {frame}/{n_frames} frames ({100*frame//n_frames}%)")
+        
+        return ax1, ax2
+    
+    # Create animation
+    anim = FuncAnimation(fig, update, frames=n_frames, 
+                        interval=1000/fps, blit=False)
+    
+    # Save video - try MP4 first, fall back to GIF if FFmpeg not available
+    print(f"\n  Saving video to: {output_path}")
+    
+    try:
+        writer = FFMpegWriter(fps=fps, bitrate=5000, codec='libx264')
+        anim.save(str(output_path), writer=writer, dpi=100)
+        print(f"✓ Video saved successfully!")
+        print(f"  Output: {output_path}")
+        print(f"  Duration: {n_frames/fps:.2f} seconds")
+        print(f"  Format: MP4")
+    except (FileNotFoundError, OSError) as e:
+        # FFmpeg not found, try GIF instead
+        print(f"  ⚠ FFmpeg not found, saving as GIF instead...")
+        gif_path = str(output_path).replace('.mp4', '.gif')
+        
+        try:
+            writer = PillowWriter(fps=fps)
+            anim.save(gif_path, writer=writer, dpi=100)
+            print(f"✓ GIF saved successfully!")
+            print(f"  Output: {gif_path}")
+            print(f"  Duration: {n_frames/fps:.2f} seconds")
+            print(f"  Format: GIF")
+            print(f"\n  Note: For MP4 format, install FFmpeg:")
+            print(f"    conda install -c conda-forge ffmpeg")
+        except Exception as gif_error:
+            print(f"✗ Error saving GIF: {gif_error}")
+            raise
+    except Exception as e:
+        print(f"✗ Error saving video: {e}")
+        raise
+    finally:
+        plt.close(fig)
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description='Generate side-by-side comparison video from existing pose3D images'
+    )
+    parser.add_argument('--user-images', required=True,
+                       help='Directory containing user pose3D images (e.g., user_videos_cache/user/pose3D)')
+    parser.add_argument('--reference-images', required=True,
+                       help='Directory containing reference pose3D images')
+    parser.add_argument('--output', default='comparison_from_images.mp4',
+                       help='Output video path')
+    parser.add_argument('--user-name', default='Your Form',
+                       help='Display name for user')
+    parser.add_argument('--reference-name', default='Correct Form',
+                       help='Display name for reference')
+    parser.add_argument('--fps', type=int, default=30,
+                       help='Frames per second')
+    
+    args = parser.parse_args()
+    
+    print("="*60)
+    print("3D POSE COMPARISON VIDEO FROM IMAGES")
+    print("="*60)
+    
+    create_comparison_video_from_images(
+        user_image_dir=args.user_images,
+        reference_image_dir=args.reference_images,
+        output_path=args.output,
+        user_video_name=args.user_name,
+        reference_name=args.reference_name,
+        fps=args.fps
+    )
+    
+    print("\n" + "="*60)
+    print("Done!")
+    print("="*60)
+
+
+if __name__ == '__main__':
+    main()
+