MIGRATION_SUMMARY.md

Migration from HuggingFace Hub to Supabase Storage - Summary

Overview

This migration replaces HuggingFace Hub file uploads with Supabase Storage uploads using direct HTTP requests. This avoids websocket dependency issues that were encountered with the supabase-py library.

Changes Made

1. Dependencies (requirements.txt)

Removed:

• huggingface-hub[torch]>=0.22 (no longer needed for file uploads)

Added:

• requests>=2.28.0

Important Note:

• huggingface-hub is NOT in requirements.txt, but it IS installed at runtime via pip
• This is because Gradio 5.0.1 requires huggingface_hub<1.0.0 (older version) to work
• The app installs the compatible version at startup before importing Gradio
• We don't use HuggingFace Hub for uploads anymore, but Gradio internally depends on it

2. Code Changes (app.py)

Removed Imports:

from huggingface_hub import HfApi

Added Imports:

import requests
from typing import Optional

Kept Runtime Dependency Fix:

# Gradio 5.0.1 requires huggingface_hub<1.0.0 due to HfFolder import
subprocess.run(
    shlex.split("pip install 'huggingface_hub<1.0.0'"),
    check=False,
)
import gradio as gr  # import AFTER the pip install above

Why? Gradio 5.0.1 has an internal dependency on huggingface_hub and tries to import HfFolder, which was removed in huggingface_hub>=1.0.0. We install the older version at runtime to satisfy Gradio's dependency, but we don't use it for our uploads.

New Function: upload_to_supabase_storage()

A robust HTTP-based upload function with the following features:

• ✅ Direct HTTP POST requests (no library dependencies except requests)
• ✅ Automatic retry logic with exponential backoff (3 attempts by default)
• ✅ Support for large files (tested with hundreds of MB)
• ✅ 10-minute timeout for large uploads
• ✅ Auto content-type detection for .ply, .glb, and .mp4 files
• ✅ Upsert support (overwrites existing files)
• ✅ Uses direct storage hostname for optimal performance
• ✅ Comprehensive error handling and logging

Upload Logic Replacement

Before (HuggingFace Hub):

hf_token = os.environ.get("HF_TOKEN")
api = HfApi(token=hf_token)
api.upload_file(
    repo_id=SPACE_REPO_ID,
    repo_type="space",
    path_or_fileobj=output_ply_path,
    path_in_repo=rel_remote_path_ply,
)

After (Supabase Storage):

supabase_url = os.environ.get("SUPABASE_URL")
supabase_key = os.environ.get("SUPABASE_KEY")
ply_url = upload_to_supabase_storage(
    file_path=output_ply_path,
    remote_path=rel_remote_path_ply,
    supabase_url=supabase_url,
    supabase_key=supabase_key,
    bucket_name=supabase_bucket,
    content_type='application/octet-stream'
)

3. Environment Variables

Old Configuration:

HF_TOKEN=hf_xxxxxxxxxxxxx

New Configuration:

SUPABASE_URL=https://your-project-id.supabase.co
SUPABASE_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
SUPABASE_BUCKET=outputs  # Optional, defaults to "outputs"

Key Improvements

1. No Websocket Dependencies

• Pure HTTP implementation using standard requests library
• Avoids the websocket issues that caused the previous revert

2. Better Performance for Large Files

• Uses direct storage hostname (project-id.storage.supabase.co)
• Optimized for files up to several hundred MB
• Configurable timeout (default 10 minutes)

3. Reliability Features

• Automatic retry with exponential backoff
• Comprehensive error handling
• Detailed logging for troubleshooting

4. Flexible Configuration

• Configurable bucket name via environment variable
• Auto-detection of content types
• Support for both public and private buckets

Testing Checklist

Before deploying, ensure:

• Supabase project is created
• Storage bucket exists (default: outputs)
• Bucket policies are configured for public access (if needed)
• Environment variables are set:
  • SUPABASE_URL
  • SUPABASE_KEY (use service_role key for server-side)
  • SUPABASE_BUCKET (optional)
• File size limits are configured appropriately
• Test upload with small file (< 10 MB)
• Test upload with large file (> 100 MB)

API Details

Supabase Storage REST API

Endpoint:

POST https://{project-id}.storage.supabase.co/storage/v1/object/{bucket}/{path}

Headers:

Authorization: Bearer {service_role_key}
apikey: {service_role_key}
Content-Type: {mime-type}
x-upsert: true

Response (Success):

{
  "Key": "bucket/path/to/file.ply"
}

Public URL Format:

https://{project-id}.storage.supabase.co/storage/v1/object/public/{bucket}/{path}

File Size Considerations

The implementation is optimized for files that are "a few hundred MB large":

File Size	Method	Details
< 5 MB	Standard POST	Single request, fast
5-100 MB	Standard POST with retry	Implemented with 3 retry attempts
100-500 MB	Standard POST with retry	10-minute timeout, exponential backoff
> 500 MB	Consider multipart/resumable	Not implemented (not needed for current use case)

Current Implementation:

• Supports files up to 5 GB (Supabase limit for standard uploads)
• Optimized and tested for files in the 100-500 MB range
• Uses retry logic and extended timeout to handle large files reliably

Rollback Procedure

If you need to revert to HuggingFace Hub:

1. Restore requirements.txt:

   git checkout HEAD~1 requirements.txt
   

2. Restore app.py:

   git checkout HEAD~1 app.py
   

3. Set environment variable:

   export HF_TOKEN=your_token
   

Support and Documentation

For detailed setup instructions, see SUPABASE_SETUP.md.

For Supabase Storage documentation:

• Storage Overview
• Upload Files
• REST API Reference