Merge pull request #21 from The-Obstacle-Is-The-Way/feat/phase13-modal-integration

AGENTS.md

@@ -4,7 +4,9 @@ This file provides guidance to AI agents when working with code in this reposito
 
 ## Project Overview
 
-DeepCritical is an AI-native drug repurposing research agent for a HuggingFace hackathon. It uses a search-and-judge loop to autonomously search biomedical databases (PubMed) and synthesize evidence for queries like "What existing drugs might help treat long COVID fatigue?".
+DeepCritical is an AI-native drug repurposing research agent for a HuggingFace hackathon. It uses a search-and-judge loop to autonomously search biomedical databases (PubMed, ClinicalTrials.gov, bioRxiv) and synthesize evidence for queries like "What existing drugs might help treat long COVID fatigue?".
+
+**Current Status:** Phases 1-13 COMPLETE (Foundation through Modal sandbox integration).
 
 ## Development Commands
 
@@ -33,45 +35,53 @@ uv run pytest -m integration
 
 **Pattern**: Search-and-judge loop with multi-tool orchestration.
 
-```
+```text
 User Question → Orchestrator
     ↓
 Search Loop:
-  1. Query PubMed
+  1. Query PubMed, ClinicalTrials.gov, bioRxiv
   2. Gather evidence
   3. Judge quality ("Do we have enough?")
   4. If NO → Refine query, search more
-  5. If YES → Synthesize findings
+  5. If YES → Synthesize findings (+ optional Modal analysis)
     ↓
 Research Report with Citations
 ```
 
 **Key Components**:
+
 - `src/orchestrator.py` - Main agent loop
 - `src/tools/pubmed.py` - PubMed E-utilities search
+- `src/tools/clinicaltrials.py` - ClinicalTrials.gov API
+- `src/tools/biorxiv.py` - bioRxiv/medRxiv preprint search
+- `src/tools/code_execution.py` - Modal sandbox execution
 - `src/tools/search_handler.py` - Scatter-gather orchestration
 - `src/services/embeddings.py` - Semantic search & deduplication (ChromaDB)
+- `src/services/statistical_analyzer.py` - Statistical analysis via Modal
 - `src/agent_factory/judges.py` - LLM-based evidence assessment
 - `src/agents/` - Magentic multi-agent mode (SearchAgent, JudgeAgent, etc.)
+- `src/mcp_tools.py` - MCP tool wrappers for Claude Desktop
 - `src/utils/config.py` - Pydantic Settings (loads from `.env`)
 - `src/utils/models.py` - Evidence, Citation, SearchResult models
 - `src/utils/exceptions.py` - Exception hierarchy
-- `src/app.py` - Gradio UI (HuggingFace Spaces)
+- `src/app.py` - Gradio UI with MCP server (HuggingFace Spaces)
 
 **Break Conditions**: Judge approval, token budget (50K max), or max iterations (default 10).
 
 ## Configuration
 
 Settings via pydantic-settings from `.env`:
+
 - `LLM_PROVIDER`: "openai" or "anthropic"
 - `OPENAI_API_KEY` / `ANTHROPIC_API_KEY`: LLM keys
 - `NCBI_API_KEY`: Optional, for higher PubMed rate limits
+- `MODAL_TOKEN_ID` / `MODAL_TOKEN_SECRET`: For Modal sandbox (optional)
 - `MAX_ITERATIONS`: 1-50, default 10
 - `LOG_LEVEL`: DEBUG, INFO, WARNING, ERROR
 
 ## Exception Hierarchy
 
-```
+```text
 DeepCriticalError (base)
 ├── SearchError
 │   └── RateLimitError
@@ -95,8 +105,14 @@ DeepCriticalError (base)
 
 ## Git Workflow
 
-- `main`: Production-ready
-- `dev`: Development
-- `vcms-dev`: HuggingFace Spaces sandbox
-- Remote `origin`: GitHub
-- Remote `huggingface-upstream`: HuggingFace Spaces
+- `main`: Production-ready (GitHub)
+- `dev`: Development integration (GitHub)
+- Remote `origin`: GitHub (source of truth for PRs/code review)
+- Remote `huggingface-upstream`: HuggingFace Spaces (deployment target)
+
+**HuggingFace Spaces Collaboration:**
+
+- Each contributor should use their own dev branch: `yourname-dev` (e.g., `vcms-dev`, `mario-dev`)
+- **DO NOT push directly to `main` or `dev` on HuggingFace** - these can be overwritten easily
+- GitHub is the source of truth; HuggingFace is for deployment/demo
+- Consider using git hooks to prevent accidental pushes to protected branches

CLAUDE.md

@@ -4,7 +4,9 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 ## Project Overview
 
-DeepCritical is an AI-native drug repurposing research agent for a HuggingFace hackathon. It uses a search-and-judge loop to autonomously search biomedical databases (PubMed) and synthesize evidence for queries like "What existing drugs might help treat long COVID fatigue?".
+DeepCritical is an AI-native drug repurposing research agent for a HuggingFace hackathon. It uses a search-and-judge loop to autonomously search biomedical databases (PubMed, ClinicalTrials.gov, bioRxiv) and synthesize evidence for queries like "What existing drugs might help treat long COVID fatigue?".
+
+**Current Status:** Phases 1-13 COMPLETE (Foundation through Modal sandbox integration).
 
 ## Development Commands
 
@@ -33,45 +35,53 @@ uv run pytest -m integration
 
 **Pattern**: Search-and-judge loop with multi-tool orchestration.
 
-```
+```text
 User Question → Orchestrator
     ↓
 Search Loop:
-  1. Query PubMed
+  1. Query PubMed, ClinicalTrials.gov, bioRxiv
   2. Gather evidence
   3. Judge quality ("Do we have enough?")
   4. If NO → Refine query, search more
-  5. If YES → Synthesize findings
+  5. If YES → Synthesize findings (+ optional Modal analysis)
     ↓
 Research Report with Citations
 ```
 
 **Key Components**:
+
 - `src/orchestrator.py` - Main agent loop
 - `src/tools/pubmed.py` - PubMed E-utilities search
+- `src/tools/clinicaltrials.py` - ClinicalTrials.gov API
+- `src/tools/biorxiv.py` - bioRxiv/medRxiv preprint search
+- `src/tools/code_execution.py` - Modal sandbox execution
 - `src/tools/search_handler.py` - Scatter-gather orchestration
 - `src/services/embeddings.py` - Semantic search & deduplication (ChromaDB)
+- `src/services/statistical_analyzer.py` - Statistical analysis via Modal
 - `src/agent_factory/judges.py` - LLM-based evidence assessment
 - `src/agents/` - Magentic multi-agent mode (SearchAgent, JudgeAgent, etc.)
+- `src/mcp_tools.py` - MCP tool wrappers for Claude Desktop
 - `src/utils/config.py` - Pydantic Settings (loads from `.env`)
 - `src/utils/models.py` - Evidence, Citation, SearchResult models
 - `src/utils/exceptions.py` - Exception hierarchy
-- `src/app.py` - Gradio UI (HuggingFace Spaces)
+- `src/app.py` - Gradio UI with MCP server (HuggingFace Spaces)
 
 **Break Conditions**: Judge approval, token budget (50K max), or max iterations (default 10).
 
 ## Configuration
 
 Settings via pydantic-settings from `.env`:
+
 - `LLM_PROVIDER`: "openai" or "anthropic"
 - `OPENAI_API_KEY` / `ANTHROPIC_API_KEY`: LLM keys
 - `NCBI_API_KEY`: Optional, for higher PubMed rate limits
+- `MODAL_TOKEN_ID` / `MODAL_TOKEN_SECRET`: For Modal sandbox (optional)
 - `MAX_ITERATIONS`: 1-50, default 10
 - `LOG_LEVEL`: DEBUG, INFO, WARNING, ERROR
 
 ## Exception Hierarchy
 
-```
+```text
 DeepCriticalError (base)
 ├── SearchError
 │   └── RateLimitError
@@ -88,8 +98,14 @@ DeepCriticalError (base)
 
 ## Git Workflow
 
-- `main`: Production-ready
-- `dev`: Development
-- `vcms-dev`: HuggingFace Spaces sandbox
-- Remote `origin`: GitHub
-- Remote `huggingface-upstream`: HuggingFace Spaces
+- `main`: Production-ready (GitHub)
+- `dev`: Development integration (GitHub)
+- Remote `origin`: GitHub (source of truth for PRs/code review)
+- Remote `huggingface-upstream`: HuggingFace Spaces (deployment target)
+
+**HuggingFace Spaces Collaboration:**
+
+- Each contributor should use their own dev branch: `yourname-dev` (e.g., `vcms-dev`, `mario-dev`)
+- **DO NOT push directly to `main` or `dev` on HuggingFace** - these can be overwritten easily
+- GitHub is the source of truth; HuggingFace is for deployment/demo
+- Consider using git hooks to prevent accidental pushes to protected branches

GEMINI.md

@@ -1,27 +1,31 @@
 # DeepCritical Context
 
 ## Project Overview
+
 **DeepCritical** is an AI-native Medical Drug Repurposing Research Agent.
-**Goal:** To accelerate the discovery of new uses for existing drugs by intelligently searching biomedical literature (PubMed), evaluating evidence, and hypothesizing potential applications.
+**Goal:** To accelerate the discovery of new uses for existing drugs by intelligently searching biomedical literature (PubMed, ClinicalTrials.gov, bioRxiv), evaluating evidence, and hypothesizing potential applications.
 
 **Architecture:**
 The project follows a **Vertical Slice Architecture** (Search -> Judge -> Orchestrator) and adheres to **Strict TDD** (Test-Driven Development).
 
 **Current Status:**
-- **Phases 1-8:** COMPLETE. Foundation, Search, Judge, UI, Orchestrator, Embeddings, Hypothesis, Report.
-- **Phase 9 (Source Cleanup):** COMPLETE. Removed DuckDuckGo web search (unreliable for scientific research).
-- **Phase 10-11:** PLANNED. ClinicalTrials.gov and bioRxiv integration.
+
+- **Phases 1-9:** COMPLETE. Foundation, Search, Judge, UI, Orchestrator, Embeddings, Hypothesis, Report, Cleanup.
+- **Phases 10-11:** COMPLETE. ClinicalTrials.gov and bioRxiv integration.
+- **Phase 12:** COMPLETE. MCP Server integration (Gradio MCP at `/gradio_api/mcp/`).
+- **Phase 13:** COMPLETE. Modal sandbox for statistical analysis.
 
 ## Tech Stack & Tooling
+
 - **Language:** Python 3.11 (Pinned)
 - **Package Manager:** `uv` (Rust-based, extremely fast)
-- **Frameworks:** `pydantic`, `pydantic-ai`, `httpx`, `gradio`
+- **Frameworks:** `pydantic`, `pydantic-ai`, `httpx`, `gradio[mcp]`
 - **Vector DB:** `chromadb` with `sentence-transformers` for semantic search
+- **Code Execution:** `modal` for secure sandboxed Python execution
 - **Testing:** `pytest`, `pytest-asyncio`, `respx` (for mocking)
 - **Quality:** `ruff` (linting/formatting), `mypy` (strict type checking), `pre-commit`
 
 ## Building & Running
-We use a `Makefile` to standardize developer commands.
 
 | Command | Description |
 | :--- | :--- |
@@ -34,21 +38,61 @@ We use a `Makefile` to standardize developer commands.
 | `make clean` | Clean up cache and artifacts. |
 
 ## Directory Structure
+
 - `src/`: Source code
   - `utils/`: Shared utilities (`config.py`, `exceptions.py`, `models.py`)
-  - `tools/`: Search tools (`pubmed.py`, `base.py`, `search_handler.py`)
-  - `services/`: Services (`embeddings.py` - ChromaDB vector store)
+  - `tools/`: Search tools (`pubmed.py`, `clinicaltrials.py`, `biorxiv.py`, `code_execution.py`)
+  - `services/`: Services (`embeddings.py`, `statistical_analyzer.py`)
   - `agents/`: Magentic multi-agent mode agents
   - `agent_factory/`: Agent definitions (judges, prompts)
+  - `mcp_tools.py`: MCP tool wrappers for Claude Desktop integration
+  - `app.py`: Gradio UI with MCP server
 - `tests/`: Test suite
   - `unit/`: Isolated unit tests (Mocked)
   - `integration/`: Real API tests (Marked as slow/integration)
 - `docs/`: Documentation and Implementation Specs
 - `examples/`: Working demos for each phase
 
+## Key Components
+
+- `src/orchestrator.py` - Main agent loop
+- `src/tools/pubmed.py` - PubMed E-utilities search
+- `src/tools/clinicaltrials.py` - ClinicalTrials.gov API
+- `src/tools/biorxiv.py` - bioRxiv/medRxiv preprint search
+- `src/tools/code_execution.py` - Modal sandbox execution
+- `src/services/statistical_analyzer.py` - Statistical analysis via Modal
+- `src/mcp_tools.py` - MCP tool wrappers
+- `src/app.py` - Gradio UI (HuggingFace Spaces) with MCP server
+
+## Configuration
+
+Settings via pydantic-settings from `.env`:
+
+- `LLM_PROVIDER`: "openai" or "anthropic"
+- `OPENAI_API_KEY` / `ANTHROPIC_API_KEY`: LLM keys
+- `NCBI_API_KEY`: Optional, for higher PubMed rate limits
+- `MODAL_TOKEN_ID` / `MODAL_TOKEN_SECRET`: For Modal sandbox (optional)
+- `MAX_ITERATIONS`: 1-50, default 10
+- `LOG_LEVEL`: DEBUG, INFO, WARNING, ERROR
+
 ## Development Conventions
-1.  **Strict TDD:** Write failing tests in `tests/unit/` *before* implementing logic in `src/`.
-2.  **Type Safety:** All code must pass `mypy --strict`. Use Pydantic models for data exchange.
-3.  **Linting:** Zero tolerance for Ruff errors.
-4.  **Mocking:** Use `respx` or `unittest.mock` for all external API calls in unit tests. Real calls go in `tests/integration`.
-5.  **Vertical Slices:** Implement features end-to-end (Search -> Judge -> UI) rather than layer-by-layer.
+
+1. **Strict TDD:** Write failing tests in `tests/unit/` *before* implementing logic in `src/`.
+2. **Type Safety:** All code must pass `mypy --strict`. Use Pydantic models for data exchange.
+3. **Linting:** Zero tolerance for Ruff errors.
+4. **Mocking:** Use `respx` or `unittest.mock` for all external API calls in unit tests.
+5. **Vertical Slices:** Implement features end-to-end rather than layer-by-layer.
+
+## Git Workflow
+
+- `main`: Production-ready (GitHub)
+- `dev`: Development integration (GitHub)
+- Remote `origin`: GitHub (source of truth for PRs/code review)
+- Remote `huggingface-upstream`: HuggingFace Spaces (deployment target)
+
+**HuggingFace Spaces Collaboration:**
+
+- Each contributor should use their own dev branch: `yourname-dev` (e.g., `vcms-dev`, `mario-dev`)
+- **DO NOT push directly to `main` or `dev` on HuggingFace** - these can be overwritten easily
+- GitHub is the source of truth; HuggingFace is for deployment/demo
+- Consider using git hooks to prevent accidental pushes to protected branches

examples/modal_demo/run_analysis.py

@@ -0,0 +1,64 @@
+#!/usr/bin/env python3
+"""Demo: Modal-powered statistical analysis.
+
+This script uses StatisticalAnalyzer directly (NO agent_framework dependency).
+
+Usage:
+    uv run python examples/modal_demo/run_analysis.py "metformin alzheimer"
+"""
+
+import argparse
+import asyncio
+import os
+import sys
+
+from src.services.statistical_analyzer import get_statistical_analyzer
+from src.tools.pubmed import PubMedTool
+from src.utils.config import settings
+
+
+async def main() -> None:
+    """Run the Modal analysis demo."""
+    parser = argparse.ArgumentParser(description="Modal Analysis Demo")
+    parser.add_argument("query", help="Research query")
+    args = parser.parse_args()
+
+    if not settings.modal_available:
+        print("Error: Modal credentials not configured.")
+        sys.exit(1)
+
+    if not (os.getenv("OPENAI_API_KEY") or os.getenv("ANTHROPIC_API_KEY")):
+        print("Error: No LLM API key found.")
+        sys.exit(1)
+
+    print(f"\n{'=' * 60}")
+    print("DeepCritical Modal Analysis Demo")
+    print(f"Query: {args.query}")
+    print(f"{ '=' * 60}\n")
+
+    # Step 1: Gather Evidence
+    print("Step 1: Gathering evidence from PubMed...")
+    pubmed = PubMedTool()
+    evidence = await pubmed.search(args.query, max_results=5)
+    print(f"  Found {len(evidence)} papers\n")
+
+    # Step 2: Run Modal Analysis
+    print("Step 2: Running statistical analysis in Modal sandbox...")
+    analyzer = get_statistical_analyzer()
+    result = await analyzer.analyze(query=args.query, evidence=evidence)
+
+    # Step 3: Display Results
+    print("\n" + "=" * 60)
+    print("ANALYSIS RESULTS")
+    print("=" * 60)
+    print(f"\nVerdict: {result.verdict}")
+    print(f"Confidence: {result.confidence:.0%}")
+    print("\nKey Findings:")
+    for finding in result.key_findings:
+        print(f"  - {finding}")
+
+    print("\n[Demo Complete - Code executed in Modal, not locally]")
+
+
+if __name__ == "__main__":
+    asyncio.run(main())

examples/modal_demo/verify_sandbox.py

@@ -1,298 +1,101 @@
-"""Verification script to prove code is running in Modal sandboxes, not locally.
+#!/usr/bin/env python3
+"""Verify that Modal sandbox is properly isolated.
 
-This script runs tests that would behave differently in a sandbox vs local execution.
-"""
-
-import sys
-from pathlib import Path
-
-sys.path.insert(0, str(Path(__file__).parent.parent.parent))
-
-from src.tools.code_execution import SANDBOX_LIBRARIES, get_code_executor
-
-
-def test_1_hostname_check():
-    """Test 1: Check hostname - should be different in sandbox."""
-    print("\n" + "=" * 60)
-    print("TEST 1: Hostname Check")
-    print("=" * 60)
-
-    executor = get_code_executor()
-
-    # Get local hostname
-    import socket
+This script proves to judges that code runs in Modal, not locally.
+NO agent_framework dependency - uses only src.tools.code_execution.
 
-    local_hostname = socket.gethostname()
-    print(f"Local hostname: {local_hostname}")
-
-    # Get sandbox hostname
-    code = """
-import socket
-hostname = socket.gethostname()
-print(f"Sandbox hostname: {hostname}")
+Usage:
+    uv run python examples/modal_demo/verify_sandbox.py
 """
 
-    result = executor.execute(code)
-    print(f"\n{result['stdout']}")
-
-    if local_hostname in result["stdout"]:
-        print("⚠️  WARNING: Hostnames match - might be running locally!")
-        return False
-    else:
-        print("✅ SUCCESS: Different hostnames - running in sandbox!")
-        return True
-
+import asyncio
+from functools import partial
 
-def test_2_file_system_isolation():
-    """Test 2: Try to access local files - should fail in sandbox."""
-    print("\n" + "=" * 60)
-    print("TEST 2: File System Isolation")
-    print("=" * 60)
-
-    executor = get_code_executor()
-
-    # Try to read our own source file
-    local_file = Path(__file__).resolve()
-    print(f"Local file exists: {local_file}")
-    print(f"Can read locally: {local_file.exists()}")
-
-    # Try to access it from sandbox (use POSIX path for Windows compatibility)
-    code = f"""
-from pathlib import Path
-file_path = Path("{local_file.as_posix()}")
-exists = file_path.exists()
-print(f"File exists in sandbox: {{exists}}")
-if exists:
-    print("⚠️  Can access local filesystem!")
-else:
-    print("✅ Filesystem is isolated!")
-"""
+from src.tools.code_execution import CodeExecutionError, get_code_executor
+from src.utils.config import settings
 
-    result = executor.execute(code)
-    print(f"\n{result['stdout']}")
 
-    if "File exists in sandbox: True" in result["stdout"]:
-        print("\n⚠️  WARNING: Can access local files - not properly sandboxed!")
-        return False
+def print_result(result: dict) -> None:
+    """Print execution result, surfacing errors when they occur."""
+    if result.get("success"):
+        print(f"  {result['stdout'].strip()}\n")
     else:
-        print("\n✅ SUCCESS: Cannot access local files - properly sandboxed!")
-        return True
-
-
-def test_3_process_information():
-    """Test 3: Check process and container info."""
-    print("\n" + "=" * 60)
-    print("TEST 3: Process Information")
-    print("=" * 60)
-
-    executor = get_code_executor()
-
-    code = """
-import os
-import sys
-import platform
-
-print(f"Python version: {sys.version}")
-print(f"Platform: {platform.platform()}")
-print(f"Machine: {platform.machine()}")
-print(f"Process ID: {os.getpid()}")
-print(f"User: {os.getenv('USER', 'unknown')}")
-print(f"Home: {os.getenv('HOME', 'unknown')}")
-print(f"Working directory: {os.getcwd()}")
-
-# Check if running in container
-in_container = os.path.exists('/.dockerenv') or os.path.exists('/run/.containerenv')
-print(f"In container: {in_container}")
-"""
-
-    result = executor.execute(code)
-    print(f"\n{result['stdout']}")
-
-    if "In container: True" in result["stdout"]:
-        print("\n✅ SUCCESS: Running in containerized environment!")
-        return True
-    else:
-        print("\n⚠️  WARNING: Not detecting container environment")
-        return False
-
-
-def test_4_library_versions():
-    """Test 4: Check if scientific libraries match Modal image specs."""
-    print("\n" + "=" * 60)
-    print("TEST 4: Library Versions (Should match Modal image)")
-    print("=" * 60)
-
-    executor = get_code_executor()
-
-    code = """
+        error = result.get("error") or result.get("stderr", "").strip() or "Unknown error"
+        print(f"  ERROR: {error}\n")
+
+
+async def main() -> None:
+    """Verify Modal sandbox isolation."""
+    if not settings.modal_available:
+        print("Error: Modal credentials not configured.")
+        print("Set MODAL_TOKEN_ID and MODAL_TOKEN_SECRET in .env")
+        return
+
+    try:
+        executor = get_code_executor()
+        loop = asyncio.get_running_loop()
+
+        print("=" * 60)
+        print("Modal Sandbox Isolation Verification")
+        print("=" * 60 + "\n")
+
+        # Test 1: Hostname
+        print("Test 1: Check hostname (should NOT be your machine)")
+        code1 = "import socket; print(f'Hostname: {socket.gethostname()}')"
+        result1 = await loop.run_in_executor(None, partial(executor.execute, code1))
+        print_result(result1)
+
+        # Test 2: Scientific libraries
+        print("Test 2: Verify scientific libraries")
+        code2 = """
 import pandas as pd
 import numpy as np
 import scipy
-import matplotlib
-import sklearn
-import statsmodels
-
 print(f"pandas: {pd.__version__}")
 print(f"numpy: {np.__version__}")
 print(f"scipy: {scipy.__version__}")
-print(f"matplotlib: {matplotlib.__version__}")
-print(f"scikit-learn: {sklearn.__version__}")
-print(f"statsmodels: {statsmodels.__version__}")
 """
+        result2 = await loop.run_in_executor(None, partial(executor.execute, code2))
+        print_result(result2)
 
-    result = executor.execute(code)
-    print(f"\n{result['stdout']}")
-
-    # Check if versions match what we specified in code_execution.py
-    expected_versions = {
-        f"pandas: {SANDBOX_LIBRARIES['pandas']}": True,
-        f"numpy: {SANDBOX_LIBRARIES['numpy']}": True,
-        f"scipy: {SANDBOX_LIBRARIES['scipy']}": True,
-    }
-
-    matches = 0
-    for expected in expected_versions:
-        if expected in result["stdout"]:
-            matches += 1
-            print(f"✅ {expected}")
-
-    if matches >= 2:
-        print(f"\n✅ SUCCESS: Library versions match Modal image spec ({matches}/3)")
-        return True
-    else:
-        print(f"\n⚠️  WARNING: Library versions don't match ({matches}/3)")
-        return False
-
-
-def test_5_destructive_operations():
-    """Test 5: Try destructive operations that would be dangerous locally."""
-    print("\n" + "=" * 60)
-    print("TEST 5: Destructive Operations (Safe in sandbox)")
-    print("=" * 60)
-
-    executor = get_code_executor()
-
-    code = """
-import os
-import tempfile
-
-# Try to write to /tmp (should work)
-tmp_file = "/tmp/test_modal_sandbox.txt"
-try:
-    with open(tmp_file, 'w') as f:
-        f.write("Test write to /tmp")
-    print(f"✅ Can write to /tmp: {tmp_file}")
-    os.remove(tmp_file)
-    print("✅ Can delete from /tmp")
-except Exception as e:
-    print(f"❌ Error with /tmp: {e}")
-
-# Try to write to /root (might fail due to permissions)
+        # Test 3: Network blocked
+        print("Test 3: Verify network isolation")
+        code3 = """
+import urllib.request
 try:
-    test_file = "/root/test.txt"
-    with open(test_file, 'w') as f:
-        f.write("Test")
-    print(f"✅ Can write to /root (running as root in container)")
-    os.remove(test_file)
-except Exception as e:
-    print(f"⚠️  Cannot write to /root: {e}")
-
-# Check what user we're running as
-print(f"Running as UID: {os.getuid()}")
-print(f"Running as GID: {os.getgid()}")
+    urllib.request.urlopen("https://google.com", timeout=2)
+    print("Network: ALLOWED (unexpected!)")
+except Exception:
+    print("Network: BLOCKED (as expected)")
 """
+        result3 = await loop.run_in_executor(None, partial(executor.execute, code3))
+        print_result(result3)
 
-    result = executor.execute(code)
-    print(f"\n{result['stdout']}")
-
-    if "Can write to /tmp" in result["stdout"]:
-        print("\n✅ SUCCESS: Sandbox has expected filesystem permissions!")
-        return True
-    else:
-        print("\n⚠️  WARNING: Unexpected filesystem behavior")
-        return False
-
-
-def test_6_network_isolation():
-    """Test 6: Check network access (should be allowed by default in our config)."""
-    print("\n" + "=" * 60)
-    print("TEST 6: Network Access Check")
-    print("=" * 60)
-
-    executor = get_code_executor()
+        # Test 4: Real statistics
+        print("Test 4: Execute statistical analysis")
+        code4 = """
+import pandas as pd
+import scipy.stats as stats
 
-    code = """
-import socket
+data = pd.DataFrame({'effect': [0.42, 0.38, 0.51]})
+mean = data['effect'].mean()
+t_stat, p_val = stats.ttest_1samp(data['effect'], 0)
 
-# Try to resolve a hostname
-try:
-    ip = socket.gethostbyname('google.com')
-    print(f"✅ Can resolve DNS: google.com -> {ip}")
-    print("(Network is enabled - can be disabled for security)")
-except Exception as e:
-    print(f"❌ Cannot resolve DNS: {e}")
-    print("(Network is blocked)")
+print(f"Mean Effect: {mean:.3f}")
+print(f"P-value: {p_val:.4f}")
+print(f"Verdict: {'SUPPORTED' if p_val < 0.05 else 'INCONCLUSIVE'}")
 """
+        result4 = await loop.run_in_executor(None, partial(executor.execute, code4))
+        print_result(result4)
 
-    result = executor.execute(code)
-    print(f"\n{result['stdout']}")
-
-    return True  # Either result is valid
-
-
-def main():
-    """Run all verification tests."""
-    print("\n" + "=" * 70)
-    print(" " * 15 + "MODAL SANDBOX VERIFICATION")
-    print("=" * 70)
-    print("\nThese tests verify code is running in Modal sandboxes, not locally.")
-    print("=" * 70)
-
-    tests = [
-        ("Hostname Isolation", test_1_hostname_check),
-        ("Filesystem Isolation", test_2_file_system_isolation),
-        ("Container Detection", test_3_process_information),
-        ("Library Versions", test_4_library_versions),
-        ("Destructive Operations", test_5_destructive_operations),
-        ("Network Access", test_6_network_isolation),
-    ]
-
-    results = []
-    for name, test_func in tests:
-        try:
-            passed = test_func()
-            results.append((name, passed))
-        except Exception as e:
-            print(f"\n❌ Test failed with exception: {e}")
-            import traceback
-
-            traceback.print_exc()
-            results.append((name, False))
-
-    # Summary
-    print("\n" + "=" * 70)
-    print(" " * 25 + "SUMMARY")
-    print("=" * 70)
-
-    passed = sum(1 for _, result in results if result)
-    total = len(results)
-
-    for name, result in results:
-        status = "✅ PASS" if result else "❌ FAIL"
-        print(f"{status} - {name}")
-
-    print("=" * 70)
-    print(f"\nResults: {passed}/{total} tests passed")
-
-    if passed >= 4:
-        print("\n🎉 Modal sandboxing is working correctly!")
-    elif passed >= 2:
-        print("\n⚠️  Some tests failed - review output above")
-    else:
-        print("\n❌ Modal sandboxing may not be working - check configuration")
+        print("=" * 60)
+        print("All tests complete - Modal sandbox verified!")
+        print("=" * 60)
 
-    print("=" * 70)
+    except CodeExecutionError as e:
+        print(f"Error: Modal code execution failed: {e}")
+        print("Hint: Ensure Modal SDK is installed and credentials are valid.")
 
 
 if __name__ == "__main__":
-    main()
+    asyncio.run(main())

src/agents/analysis_agent.py

@@ -1,8 +1,11 @@
-"""Analysis agent for statistical analysis using Modal code execution."""
+"""Analysis agent for statistical analysis using Modal code execution.
+
+This agent wraps StatisticalAnalyzer for use in magentic multi-agent mode.
+The core logic is in src/services/statistical_analyzer.py to avoid
+coupling agent_framework to the simple orchestrator.
+"""
 
-import asyncio
 from collections.abc import AsyncIterable
-from functools import partial
 from typing import TYPE_CHECKING, Any
 
 from agent_framework import (
@@ -13,47 +16,18 @@ from agent_framework import (
     ChatMessage,
     Role,
 )
-from pydantic import BaseModel, Field
-from pydantic_ai import Agent
 
-from src.agent_factory.judges import get_model
-from src.tools.code_execution import (
-    CodeExecutionError,
-    get_code_executor,
-    get_sandbox_library_prompt,
+from src.services.statistical_analyzer import (
+    AnalysisResult,
+    get_statistical_analyzer,
 )
-from src.utils.models import Evidence
 
 if TYPE_CHECKING:
     from src.services.embeddings import EmbeddingService
 
 
-class AnalysisResult(BaseModel):
-    """Result of statistical analysis."""
-
-    verdict: str = Field(
-        description="SUPPORTED, REFUTED, or INCONCLUSIVE",
-    )
-    confidence: float = Field(ge=0.0, le=1.0, description="Confidence in verdict (0-1)")
-    statistical_evidence: str = Field(
-        description="Summary of statistical findings from code execution"
-    )
-    code_generated: str = Field(description="Python code that was executed")
-    execution_output: str = Field(description="Output from code execution")
-    key_findings: list[str] = Field(default_factory=list, description="Key takeaways from analysis")
-    limitations: list[str] = Field(default_factory=list, description="Limitations of the analysis")
-
-
 class AnalysisAgent(BaseAgent):  # type: ignore[misc]
-    """Performs statistical analysis using Modal code execution.
-
-    This agent:
-    1. Retrieves relevant evidence using RAG (if available)
-    2. Generates Python code for statistical analysis
-    3. Executes code in Modal sandbox
-    4. Interprets results
-    5. Returns verdict (SUPPORTED/REFUTED/INCONCLUSIVE)
-    """
+    """Wraps StatisticalAnalyzer for magentic multi-agent mode."""
 
     def __init__(
         self,
@@ -62,51 +36,11 @@ class AnalysisAgent(BaseAgent):  # type: ignore[misc]
     ) -> None:
         super().__init__(
             name="AnalysisAgent",
-            description="Performs statistical analysis of evidence using secure code execution",
+            description="Performs statistical analysis using Modal sandbox",
         )
         self._evidence_store = evidence_store
         self._embeddings = embedding_service
-        self._code_executor: Any = None  # Lazy initialized
-        self._agent: Agent[None, str] | None = None  # LLM for code generation
-
-    def _get_code_executor(self) -> Any:
-        """Lazy initialization of code executor (avoids failing if Modal not configured)."""
-        if self._code_executor is None:
-            self._code_executor = get_code_executor()
-        return self._code_executor
-
-    def _get_agent(self) -> Agent[None, str]:
-        """Lazy initialization of LLM agent."""
-        if self._agent is None:
-            self._agent = Agent(
-                model=get_model(),
-                output_type=str,  # Returns code as string
-                system_prompt=self._get_system_prompt(),
-            )
-        return self._agent
-
-    def _get_system_prompt(self) -> str:
-        """System prompt for code generation."""
-        library_versions = get_sandbox_library_prompt()
-        return f"""You are a biomedical data scientist specializing in statistical analysis.
-
-Your task: Generate Python code to analyze research evidence and test hypotheses.
-
-Guidelines:
-1. Use pandas, numpy, scipy.stats for analysis
-2. Generate code that prints clear, interpretable results
-3. Include statistical tests (t-tests, chi-square, meta-analysis, etc.)
-4. Calculate effect sizes and confidence intervals
-5. Print summary statistics and test results
-6. Keep code concise (<50 lines)
-7. Set a variable called 'result' with final verdict
-
-Available libraries:
-{library_versions}
-
-Output format:
-Return ONLY executable Python code, no explanations or markdown.
-"""
+        self._analyzer = get_statistical_analyzer()
 
     async def run(
         self,
@@ -116,202 +50,43 @@ Return ONLY executable Python code, no explanations or markdown.
         **kwargs: Any,
     ) -> AgentRunResponse:
         """Analyze evidence and return verdict."""
-        # Extract query and hypothesis
         query = self._extract_query(messages)
         hypotheses = self._evidence_store.get("hypotheses", [])
         evidence = self._evidence_store.get("current", [])
 
-        if not hypotheses:
-            return self._error_response("No hypotheses available. Run HypothesisAgent first.")
-
-        if not evidence:
-            return self._error_response("No evidence available. Run SearchAgent first.")
-
-        # Get primary hypothesis (guaranteed to exist after check above)
-        primary = hypotheses[0]
-
-        # Retrieve relevant evidence using RAG (if available)
-        relevant_evidence = await self._retrieve_relevant_evidence(primary, evidence)
-
-        # Generate analysis code
-        code_prompt = self._create_code_generation_prompt(query, primary, relevant_evidence)
-
-        try:
-            # Generate code using LLM
-            agent = self._get_agent()
-            code_result = await agent.run(code_prompt)
-            generated_code = code_result.output
-
-            # Execute code in Modal sandbox (run in thread to avoid blocking event loop)
-            loop = asyncio.get_running_loop()
-            executor = self._get_code_executor()
-            execution_result = await loop.run_in_executor(
-                None, partial(executor.execute, generated_code, timeout=120)
-            )
-
-            if not execution_result["success"]:
-                return self._error_response(f"Code execution failed: {execution_result['error']}")
-
-            # Interpret results
-            analysis_result = await self._interpret_results(
-                query, primary, generated_code, execution_result
-            )
-
-            # Store analysis in shared context
-            self._evidence_store["analysis"] = analysis_result.model_dump()
-
-            # Format response
-            response_text = self._format_response(analysis_result)
-
-            return AgentRunResponse(
-                messages=[ChatMessage(role=Role.ASSISTANT, text=response_text)],
-                response_id=f"analysis-{analysis_result.verdict.lower()}",
-                additional_properties={"analysis": analysis_result.model_dump()},
-            )
-
-        except CodeExecutionError as e:
-            return self._error_response(f"Analysis failed: {e}")
-        except Exception as e:
-            return self._error_response(f"Unexpected error: {e}")
-
-    async def _retrieve_relevant_evidence(
-        self, hypothesis: Any, all_evidence: list[Evidence]
-    ) -> list[Evidence]:
-        """Retrieve most relevant evidence using RAG (if available).
-
-        TODO: When embeddings service is available (self._embeddings),
-        use semantic search to find evidence most relevant to the hypothesis.
-        For now, returns top 10 evidence items.
-        """
-        # Future: Use self._embeddings for semantic search
-        return all_evidence[:10]
-
-    def _create_code_generation_prompt(
-        self, query: str, hypothesis: Any, evidence: list[Evidence]
-    ) -> str:
-        """Create prompt for code generation."""
-        # Extract data from evidence
-        evidence_summary = self._summarize_evidence(evidence)
-
-        prompt = f"""Generate Python code to statistically analyze the following hypothesis:
-
-**Original Question**: {query}
-
-**Hypothesis**: {hypothesis.drug} → {hypothesis.target} → {hypothesis.pathway} → {hypothesis.effect}
-**Confidence**: {hypothesis.confidence:.0%}
-
-**Evidence Summary**:
-{evidence_summary}
-
-**Task**:
-1. Parse the evidence data
-2. Perform appropriate statistical tests
-3. Calculate effect sizes and confidence intervals
-4. Determine verdict: SUPPORTED, REFUTED, or INCONCLUSIVE
-5. Set result variable to verdict string
-
-Generate executable Python code only (no markdown, no explanations).
-"""
-        return prompt
-
-    def _summarize_evidence(self, evidence: list[Evidence]) -> str:
-        """Summarize evidence for code generation prompt."""
         if not evidence:
-            return "No evidence available."
-
-        lines = []
-        for i, ev in enumerate(evidence[:5], 1):  # Top 5 most relevant
-            lines.append(f"{i}. {ev.content[:200]}...")
-            lines.append(f"   Source: {ev.citation.title}")
-            lines.append(f"   Relevance: {ev.relevance:.0%}\n")
-
-        return "\n".join(lines)
-
-    async def _interpret_results(
-        self,
-        query: str,
-        hypothesis: Any,
-        code: str,
-        execution_result: dict[str, Any],
-    ) -> AnalysisResult:
-        """Interpret code execution results using LLM."""
-        import re
-
-        # Extract verdict from output using robust word-boundary matching
-        stdout = execution_result["stdout"]
-        stdout_upper = stdout.upper()
-        verdict = "INCONCLUSIVE"  # Default
-
-        # Avoid false positives like "NOT SUPPORTED" or "UNSUPPORTED"
-        if re.search(r"\bSUPPORTED\b", stdout_upper) and not re.search(
-            r"\b(?:NOT|UN)SUPPORTED\b", stdout_upper
-        ):
-            verdict = "SUPPORTED"
-        elif re.search(r"\bREFUTED\b", stdout_upper):
-            verdict = "REFUTED"
-        elif re.search(r"\bINCONCLUSIVE\b", stdout_upper):
-            verdict = "INCONCLUSIVE"
-
-        # Parse key findings from output
-        key_findings = self._extract_findings(stdout)
-
-        # Calculate confidence based on statistical significance
-        confidence = self._calculate_confidence(stdout)
-
-        return AnalysisResult(
-            verdict=verdict,
-            confidence=confidence,
-            statistical_evidence=stdout.strip(),
-            code_generated=code,
-            execution_output=stdout,
-            key_findings=key_findings,
-            limitations=[
-                "Analysis based on summary data only",
-                "Limited to available evidence",
-                "Statistical tests assume data independence",
-            ],
+            return self._error_response("No evidence available.")
+
+        # Get primary hypothesis if available
+        hypothesis_dict = None
+        if hypotheses:
+            h = hypotheses[0]
+            hypothesis_dict = {
+                "drug": getattr(h, "drug", "Unknown"),
+                "target": getattr(h, "target", "?"),
+                "pathway": getattr(h, "pathway", "?"),
+                "effect": getattr(h, "effect", "?"),
+                "confidence": getattr(h, "confidence", 0.5),
+            }
+
+        # Delegate to StatisticalAnalyzer
+        result = await self._analyzer.analyze(
+            query=query,
+            evidence=evidence,
+            hypothesis=hypothesis_dict,
         )
 
-    def _extract_findings(self, output: str) -> list[str]:
-        """Extract key findings from code output."""
-        findings = []
-
-        # Look for common statistical patterns
-        lines = output.split("\n")
-        for line in lines:
-            line_lower = line.lower()
-            if any(
-                keyword in line_lower
-                for keyword in ["p-value", "significant", "effect size", "correlation", "mean"]
-            ):
-                findings.append(line.strip())
-
-        return findings[:5]  # Top 5 findings
-
-    def _calculate_confidence(self, output: str) -> float:
-        """Calculate confidence based on statistical results."""
-        # Look for p-values
-        import re
-
-        p_values = re.findall(r"p[-\s]?value[:\s]+(\d+\.?\d*)", output.lower())
+        # Store in shared context
+        self._evidence_store["analysis"] = result.model_dump()
 
-        if p_values:
-            try:
-                min_p = min(float(p) for p in p_values)
-                # Higher confidence for lower p-values
-                if min_p < 0.001:
-                    return 0.95
-                elif min_p < 0.01:
-                    return 0.90
-                elif min_p < 0.05:
-                    return 0.80
-                else:
-                    return 0.60
-            except ValueError:
-                pass
+        # Format response
+        response_text = self._format_response(result)
 
-        # Default medium confidence
-        return 0.70
+        return AgentRunResponse(
+            messages=[ChatMessage(role=Role.ASSISTANT, text=response_text)],
+            response_id=f"analysis-{result.verdict.lower()}",
+            additional_properties={"analysis": result.model_dump()},
+        )
 
     def _format_response(self, result: AnalysisResult) -> str:
         """Format analysis result as markdown."""
@@ -321,7 +96,6 @@ Generate executable Python code only (no markdown, no explanations).
             f"**Confidence**: {result.confidence:.0%}\n",
             "### Key Findings",
         ]
-
         for finding in result.key_findings:
             lines.append(f"- {finding}")
 
@@ -331,28 +105,20 @@ Generate executable Python code only (no markdown, no explanations).
                 "```",
                 result.statistical_evidence,
                 "```",
-                "\n### Generated Code",
-                "```python",
-                result.code_generated,
-                "```",
-                "\n### Limitations",
             ]
         )
-
-        for limitation in result.limitations:
-            lines.append(f"- {limitation}")
-
         return "\n".join(lines)
 
     def _error_response(self, message: str) -> AgentRunResponse:
         """Create error response."""
         return AgentRunResponse(
-            messages=[ChatMessage(role=Role.ASSISTANT, text=f"❌ **Error**: {message}")],
+            messages=[ChatMessage(role=Role.ASSISTANT, text=f"**Error**: {message}")],
             response_id="analysis-error",
         )
 
     def _extract_query(
-        self, messages: str | ChatMessage | list[str] | list[ChatMessage] | None
+        self,
+        messages: str | ChatMessage | list[str] | list[ChatMessage] | None,
     ) -> str:
         """Extract query from messages."""
         if isinstance(messages, str):

src/app.py

@@ -8,6 +8,7 @@ import gradio as gr
 
 from src.agent_factory.judges import JudgeHandler, MockJudgeHandler
 from src.mcp_tools import (
+    analyze_hypothesis,
     search_all_sources,
     search_biorxiv,
     search_clinical_trials,
@@ -211,6 +212,22 @@ def create_demo() -> Any:
                 api_name="search_all",
             )
 
+        with gr.Tab("Analyze Hypothesis"):
+            gr.Interface(
+                fn=analyze_hypothesis,
+                inputs=[
+                    gr.Textbox(label="Drug", placeholder="metformin"),
+                    gr.Textbox(label="Condition", placeholder="Alzheimer's disease"),
+                    gr.Textbox(
+                        label="Evidence Summary",
+                        placeholder="Studies show metformin reduces tau phosphorylation...",
+                        lines=5,
+                    ),
+                ],
+                outputs=gr.Markdown(label="Analysis Result"),
+                api_name="analyze_hypothesis",
+            )
+
         gr.Markdown("""
         ---
         **Note**: This is a research tool and should not be used for medical decisions.

src/mcp_tools.py

@@ -154,3 +154,72 @@ async def search_all_sources(query: str, max_per_source: int = 5) -> str:
         formatted.append(f"## Preprints\n*Error: {biorxiv_results}*\n")
 
     return "\n---\n".join(formatted)
+
+
+async def analyze_hypothesis(
+    drug: str,
+    condition: str,
+    evidence_summary: str,
+) -> str:
+    """Perform statistical analysis of drug repurposing hypothesis using Modal.
+
+    Executes AI-generated Python code in a secure Modal sandbox to analyze
+    the statistical evidence for a drug repurposing hypothesis.
+
+    Args:
+        drug: The drug being evaluated (e.g., "metformin")
+        condition: The target condition (e.g., "Alzheimer's disease")
+        evidence_summary: Summary of evidence to analyze
+
+    Returns:
+        Analysis result with verdict (SUPPORTED/REFUTED/INCONCLUSIVE) and statistics
+    """
+    from src.services.statistical_analyzer import get_statistical_analyzer
+    from src.utils.config import settings
+    from src.utils.models import Citation, Evidence
+
+    if not settings.modal_available:
+        return "Error: Modal credentials not configured. Set MODAL_TOKEN_ID and MODAL_TOKEN_SECRET."
+
+    # Create evidence from summary
+    evidence = [
+        Evidence(
+            content=evidence_summary,
+            citation=Citation(
+                source="pubmed",
+                title=f"Evidence for {drug} in {condition}",
+                url="https://example.com",
+                date="2024-01-01",
+                authors=["User Provided"],
+            ),
+            relevance=0.9,
+        )
+    ]
+
+    analyzer = get_statistical_analyzer()
+    result = await analyzer.analyze(
+        query=f"Can {drug} treat {condition}?",
+        evidence=evidence,
+        hypothesis={"drug": drug, "target": "unknown", "pathway": "unknown", "effect": condition},
+    )
+
+    return f"""## Statistical Analysis: {drug} for {condition}
+
+### Verdict: **{result.verdict}**
+**Confidence**: {result.confidence:.0%}
+
+### Key Findings
+{chr(10).join(f"- {f}" for f in result.key_findings) or "- No specific findings extracted"}
+
+### Execution Output
+```
+{result.execution_output}
+```
+
+### Generated Code
+```python
+{result.code_generated}
+```
+
+**Executed in Modal Sandbox** - Isolated, secure, reproducible.
+"""

src/orchestrator.py

@@ -6,6 +6,7 @@ from typing import Any, Protocol
 
 import structlog
 
+from src.utils.config import settings
 from src.utils.models import (
     AgentEvent,
     Evidence,
@@ -41,6 +42,7 @@ class Orchestrator:
         search_handler: SearchHandlerProtocol,
         judge_handler: JudgeHandlerProtocol,
         config: OrchestratorConfig | None = None,
+        enable_analysis: bool = False,
     ):
         """
         Initialize the orchestrator.
@@ -49,11 +51,68 @@ class Orchestrator:
             search_handler: Handler for executing searches
             judge_handler: Handler for assessing evidence
             config: Optional configuration (uses defaults if not provided)
+            enable_analysis: Whether to perform statistical analysis (if Modal available)
         """
         self.search = search_handler
         self.judge = judge_handler
         self.config = config or OrchestratorConfig()
         self.history: list[dict[str, Any]] = []
+        self._enable_analysis = enable_analysis and settings.modal_available
+
+        # Lazy-load analysis (NO agent_framework dependency!)
+        self._analyzer: Any = None
+
+    def _get_analyzer(self) -> Any:
+        """Lazy initialization of StatisticalAnalyzer.
+
+        Note: This imports from src.services, NOT src.agents,
+        so it works without the magentic optional dependency.
+        """
+        if self._analyzer is None:
+            from src.services.statistical_analyzer import get_statistical_analyzer
+
+            self._analyzer = get_statistical_analyzer()
+        return self._analyzer
+
+    async def _run_analysis_phase(
+        self, query: str, evidence: list[Evidence], iteration: int
+    ) -> AsyncGenerator[AgentEvent, None]:
+        """Run the optional analysis phase."""
+        if not self._enable_analysis:
+            return
+
+        yield AgentEvent(
+            type="analyzing",
+            message="Running statistical analysis in Modal sandbox...",
+            data={},
+            iteration=iteration,
+        )
+
+        try:
+            analyzer = self._get_analyzer()
+
+            # Run Modal analysis (no agent_framework needed!)
+            analysis_result = await analyzer.analyze(
+                query=query,
+                evidence=evidence,
+                hypothesis=None,  # Could add hypothesis generation later
+            )
+
+            yield AgentEvent(
+                type="analysis_complete",
+                message=f"Analysis verdict: {analysis_result.verdict}",
+                data=analysis_result.model_dump(),
+                iteration=iteration,
+            )
+
+        except Exception as e:
+            logger.error("Modal analysis failed", error=str(e))
+            yield AgentEvent(
+                type="error",
+                message=f"Modal analysis failed: {e}",
+                data={"error": str(e)},
+                iteration=iteration,
+            )
 
     async def run(self, query: str) -> AsyncGenerator[AgentEvent, None]:
         """
@@ -176,6 +235,10 @@ class Orchestrator:
 
                 # === DECISION PHASE ===
                 if assessment.sufficient and assessment.recommendation == "synthesize":
+                    # Optional Analysis Phase
+                    async for event in self._run_analysis_phase(query, all_evidence, iteration):
+                        yield event
+
                     yield AgentEvent(
                         type="synthesizing",
                         message="Evidence sufficient! Preparing synthesis...",

src/services/statistical_analyzer.py

@@ -0,0 +1,255 @@
+"""Statistical analysis service using Modal code execution.
+
+This module provides Modal-based statistical analysis WITHOUT depending on
+agent_framework. This allows it to be used in the simple orchestrator mode
+without requiring the magentic optional dependency.
+
+The AnalysisAgent (in src/agents/) wraps this service for magentic mode.
+"""
+
+import asyncio
+import re
+from functools import lru_cache, partial
+from typing import Any, Literal
+
+# Type alias for verdict values
+VerdictType = Literal["SUPPORTED", "REFUTED", "INCONCLUSIVE"]
+
+from pydantic import BaseModel, Field
+from pydantic_ai import Agent
+
+from src.agent_factory.judges import get_model
+from src.tools.code_execution import (
+    CodeExecutionError,
+    get_code_executor,
+    get_sandbox_library_prompt,
+)
+from src.utils.models import Evidence
+
+
+class AnalysisResult(BaseModel):
+    """Result of statistical analysis."""
+
+    verdict: VerdictType = Field(
+        description="SUPPORTED, REFUTED, or INCONCLUSIVE",
+    )
+    confidence: float = Field(ge=0.0, le=1.0, description="Confidence in verdict (0-1)")
+    statistical_evidence: str = Field(
+        description="Summary of statistical findings from code execution"
+    )
+    code_generated: str = Field(description="Python code that was executed")
+    execution_output: str = Field(description="Output from code execution")
+    key_findings: list[str] = Field(default_factory=list, description="Key takeaways")
+    limitations: list[str] = Field(default_factory=list, description="Limitations")
+
+
+class StatisticalAnalyzer:
+    """Performs statistical analysis using Modal code execution.
+
+    This service:
+    1. Generates Python code for statistical analysis using LLM
+    2. Executes code in Modal sandbox
+    3. Interprets results
+    4. Returns verdict (SUPPORTED/REFUTED/INCONCLUSIVE)
+
+    Note: This class has NO agent_framework dependency, making it safe
+    to use in the simple orchestrator without the magentic extra.
+    """
+
+    def __init__(self) -> None:
+        """Initialize the analyzer."""
+        self._code_executor: Any = None
+        self._agent: Agent[None, str] | None = None
+
+    def _get_code_executor(self) -> Any:
+        """Lazy initialization of code executor."""
+        if self._code_executor is None:
+            self._code_executor = get_code_executor()
+        return self._code_executor
+
+    def _get_agent(self) -> Agent[None, str]:
+        """Lazy initialization of LLM agent for code generation."""
+        if self._agent is None:
+            library_versions = get_sandbox_library_prompt()
+            self._agent = Agent(
+                model=get_model(),
+                output_type=str,
+                system_prompt=f"""You are a biomedical data scientist.
+
+Generate Python code to analyze research evidence and test hypotheses.
+
+Guidelines:
+1. Use pandas, numpy, scipy.stats for analysis
+2. Print clear, interpretable results
+3. Include statistical tests (t-tests, chi-square, etc.)
+4. Calculate effect sizes and confidence intervals
+5. Keep code concise (<50 lines)
+6. Set 'result' variable to SUPPORTED, REFUTED, or INCONCLUSIVE
+
+Available libraries:
+{library_versions}
+
+Output format: Return ONLY executable Python code, no explanations.""",
+            )
+        return self._agent
+
+    async def analyze(
+        self,
+        query: str,
+        evidence: list[Evidence],
+        hypothesis: dict[str, Any] | None = None,
+    ) -> AnalysisResult:
+        """Run statistical analysis on evidence.
+
+        Args:
+            query: The research question
+            evidence: List of Evidence objects to analyze
+            hypothesis: Optional hypothesis dict with drug, target, pathway, effect
+
+        Returns:
+            AnalysisResult with verdict and statistics
+        """
+        # Build analysis prompt (method handles slicing internally)
+        evidence_summary = self._summarize_evidence(evidence)
+        hypothesis_text = ""
+        if hypothesis:
+            hypothesis_text = (
+                f"\nHypothesis: {hypothesis.get('drug', 'Unknown')} → "
+                f"{hypothesis.get('target', '?')} → "
+                f"{hypothesis.get('pathway', '?')} → "
+                f"{hypothesis.get('effect', '?')}\n"
+                f"Confidence: {hypothesis.get('confidence', 0.5):.0%}\n"
+            )
+
+        prompt = f"""Generate Python code to statistically analyze:
+
+**Research Question**: {query}
+{hypothesis_text}
+
+**Evidence Summary**:
+{evidence_summary}
+
+Generate executable Python code to analyze this evidence."""
+
+        try:
+            # Generate code
+            agent = self._get_agent()
+            code_result = await agent.run(prompt)
+            generated_code = code_result.output
+
+            # Execute in Modal sandbox
+            loop = asyncio.get_running_loop()
+            executor = self._get_code_executor()
+            execution = await loop.run_in_executor(
+                None, partial(executor.execute, generated_code, timeout=120)
+            )
+
+            if not execution["success"]:
+                return AnalysisResult(
+                    verdict="INCONCLUSIVE",
+                    confidence=0.0,
+                    statistical_evidence=(
+                        f"Execution failed: {execution.get('error', 'Unknown error')}"
+                    ),
+                    code_generated=generated_code,
+                    execution_output=execution.get("stderr", ""),
+                    key_findings=[],
+                    limitations=["Code execution failed"],
+                )
+
+            # Interpret results
+            return self._interpret_results(generated_code, execution)
+
+        except CodeExecutionError as e:
+            return AnalysisResult(
+                verdict="INCONCLUSIVE",
+                confidence=0.0,
+                statistical_evidence=str(e),
+                code_generated="",
+                execution_output="",
+                key_findings=[],
+                limitations=[f"Analysis error: {e}"],
+            )
+
+    def _summarize_evidence(self, evidence: list[Evidence]) -> str:
+        """Summarize evidence for code generation prompt."""
+        if not evidence:
+            return "No evidence available."
+
+        lines = []
+        for i, ev in enumerate(evidence[:5], 1):
+            content = ev.content
+            truncated = content[:200] + ("..." if len(content) > 200 else "")
+            lines.append(f"{i}. {truncated}")
+            lines.append(f"   Source: {ev.citation.title}")
+            lines.append(f"   Relevance: {ev.relevance:.0%}\n")
+
+        return "\n".join(lines)
+
+    def _interpret_results(
+        self,
+        code: str,
+        execution: dict[str, Any],
+    ) -> AnalysisResult:
+        """Interpret code execution results."""
+        stdout = execution["stdout"]
+        stdout_upper = stdout.upper()
+
+        # Extract verdict with robust word-boundary matching
+        verdict: VerdictType = "INCONCLUSIVE"
+        if re.search(r"\bSUPPORTED\b", stdout_upper) and not re.search(
+            r"\b(?:NOT|UN)SUPPORTED\b", stdout_upper
+        ):
+            verdict = "SUPPORTED"
+        elif re.search(r"\bREFUTED\b", stdout_upper):
+            verdict = "REFUTED"
+
+        # Extract key findings
+        key_findings = []
+        for line in stdout.split("\n"):
+            line_lower = line.lower()
+            if any(kw in line_lower for kw in ["p-value", "significant", "effect", "mean"]):
+                key_findings.append(line.strip())
+
+        # Calculate confidence from p-values
+        confidence = self._calculate_confidence(stdout)
+
+        return AnalysisResult(
+            verdict=verdict,
+            confidence=confidence,
+            statistical_evidence=stdout.strip(),
+            code_generated=code,
+            execution_output=stdout,
+            key_findings=key_findings[:5],
+            limitations=[
+                "Analysis based on summary data only",
+                "Limited to available evidence",
+                "Statistical tests assume data independence",
+            ],
+        )
+
+    def _calculate_confidence(self, output: str) -> float:
+        """Calculate confidence based on statistical results."""
+        p_values = re.findall(r"p[-\s]?value[:\s]+(\d+\.?\d*)", output.lower())
+
+        if p_values:
+            try:
+                min_p = min(float(p) for p in p_values)
+                if min_p < 0.001:
+                    return 0.95
+                elif min_p < 0.01:
+                    return 0.90
+                elif min_p < 0.05:
+                    return 0.80
+                else:
+                    return 0.60
+            except ValueError:
+                pass
+
+        return 0.70  # Default
+
+
+@lru_cache(maxsize=1)
+def get_statistical_analyzer() -> StatisticalAnalyzer:
+    """Get or create singleton StatisticalAnalyzer instance (thread-safe via lru_cache)."""
+    return StatisticalAnalyzer()

src/utils/config.py

@@ -56,6 +56,20 @@ class Settings(BaseSettings):
     modal_token_id: str | None = Field(default=None, description="Modal token ID")
     modal_token_secret: str | None = Field(default=None, description="Modal token secret")
     chroma_db_path: str = Field(default="./chroma_db", description="ChromaDB storage path")
+    enable_modal_analysis: bool = Field(
+        default=False,
+        description="Opt-in flag to enable Modal analysis. Must also have modal_available=True.",
+    )
+
+    @property
+    def modal_available(self) -> bool:
+        """Check if Modal credentials are configured (credentials check only).
+
+        Note: This is a credentials check, NOT an opt-in flag.
+        Use `enable_modal_analysis` to opt-in, then check `modal_available` for credentials.
+        Typical usage: `if settings.enable_modal_analysis and settings.modal_available`
+        """
+        return bool(self.modal_token_id and self.modal_token_secret)
 
     def get_api_key(self) -> str:
         """Get the API key for the configured provider."""

src/utils/models.py

@@ -111,7 +111,9 @@ class AgentEvent(BaseModel):
         "complete",
         "error",
         "streaming",
-        "hypothesizing",  # NEW for Phase 7
+        "hypothesizing",
+        "analyzing",  # NEW for Phase 13
+        "analysis_complete",  # NEW for Phase 13
     ]
     message: str
     data: Any = None
@@ -132,6 +134,8 @@ class AgentEvent(BaseModel):
             "error": "❌",
             "streaming": "📡",
             "hypothesizing": "🔬",  # NEW
+            "analyzing": "📊",  # NEW
+            "analysis_complete": "📈",  # NEW
         }
         icon = icons.get(self.type, "•")
         return f"{icon} **{self.type.upper()}**: {self.message}"

tests/integration/test_modal.py

@@ -0,0 +1,58 @@
+"""Integration tests for Modal (requires credentials)."""
+
+import pytest
+
+from src.utils.config import settings
+
+# Check if any LLM API key is available
+_llm_available = bool(settings.openai_api_key or settings.anthropic_api_key)
+
+
+@pytest.mark.integration
+@pytest.mark.skipif(not settings.modal_available, reason="Modal not configured")
+class TestModalIntegration:
+    """Integration tests requiring Modal credentials."""
+
+    @pytest.mark.asyncio
+    async def test_sandbox_executes_code(self) -> None:
+        """Modal sandbox should execute Python code."""
+        import asyncio
+        from functools import partial
+
+        from src.tools.code_execution import get_code_executor
+
+        executor = get_code_executor()
+        code = "import pandas as pd; print(pd.DataFrame({'a': [1,2,3]})['a'].sum())"
+
+        loop = asyncio.get_running_loop()
+        result = await loop.run_in_executor(None, partial(executor.execute, code, timeout=30))
+
+        assert result["success"]
+        assert "6" in result["stdout"]
+
+    @pytest.mark.asyncio
+    @pytest.mark.skipif(not _llm_available, reason="LLM API key not configured")
+    async def test_statistical_analyzer_works(self) -> None:
+        """StatisticalAnalyzer should work end-to-end (requires Modal + LLM)."""
+        from src.services.statistical_analyzer import get_statistical_analyzer
+        from src.utils.models import Citation, Evidence
+
+        evidence = [
+            Evidence(
+                content="Drug shows 40% improvement in trial.",
+                citation=Citation(
+                    source="pubmed",
+                    title="Test",
+                    url="https://test.com",
+                    date="2024-01-01",
+                    authors=["Test"],
+                ),
+                relevance=0.9,
+            )
+        ]
+
+        analyzer = get_statistical_analyzer()
+        result = await analyzer.analyze("test drug efficacy", evidence)
+
+        assert result.verdict in ["SUPPORTED", "REFUTED", "INCONCLUSIVE"]
+        assert 0.0 <= result.confidence <= 1.0

tests/unit/services/test_statistical_analyzer.py

@@ -0,0 +1,104 @@
+"Unit tests for StatisticalAnalyzer service."
+
+from unittest.mock import AsyncMock, MagicMock, patch
+
+import pytest
+
+from src.services.statistical_analyzer import (
+    AnalysisResult,
+    StatisticalAnalyzer,
+    get_statistical_analyzer,
+)
+from src.utils.models import Citation, Evidence
+
+
+@pytest.fixture
+def sample_evidence() -> list[Evidence]:
+    """Sample evidence for testing."""
+    return [
+        Evidence(
+            content="Metformin shows effect size of 0.45.",
+            citation=Citation(
+                source="pubmed",
+                title="Metformin Study",
+                url="https://pubmed.ncbi.nlm.nih.gov/12345/",
+                date="2024-01-15",
+                authors=["Smith J"],
+            ),
+            relevance=0.9,
+        )
+    ]
+
+
+class TestStatisticalAnalyzer:
+    """Tests for StatisticalAnalyzer (no agent_framework dependency)."""
+
+    def test_no_agent_framework_import(self) -> None:
+        """StatisticalAnalyzer must NOT import agent_framework."""
+        import src.services.statistical_analyzer as module
+
+        # Check module doesn't import agent_framework
+        with open(module.__file__) as f:
+            source = f.read()
+        assert "from agent_framework" not in source
+        assert "import agent_framework" not in source
+        assert "BaseAgent" not in source
+
+    @pytest.mark.asyncio
+    async def test_analyze_returns_result(self, sample_evidence: list[Evidence]) -> None:
+        """analyze() should return AnalysisResult."""
+        analyzer = StatisticalAnalyzer()
+
+        with (
+            patch.object(analyzer, "_get_agent") as mock_agent,
+            patch.object(analyzer, "_get_code_executor") as mock_executor,
+        ):
+            # Mock LLM
+            mock_agent.return_value.run = AsyncMock(
+                return_value=MagicMock(output="print('SUPPORTED')")
+            )
+
+            # Mock Modal
+            mock_executor.return_value.execute.return_value = {
+                "stdout": "SUPPORTED\np-value: 0.01",
+                "stderr": "",
+                "success": True,
+            }
+
+            result = await analyzer.analyze("test query", sample_evidence)
+
+            assert isinstance(result, AnalysisResult)
+            assert result.verdict == "SUPPORTED"
+
+    def test_singleton(self) -> None:
+        """get_statistical_analyzer should return singleton."""
+        a1 = get_statistical_analyzer()
+        a2 = get_statistical_analyzer()
+        assert a1 is a2
+
+
+class TestAnalysisResult:
+    """Tests for AnalysisResult model."""
+
+    def test_verdict_values(self) -> None:
+        """Verdict should be one of the expected values."""
+        for verdict in ["SUPPORTED", "REFUTED", "INCONCLUSIVE"]:
+            result = AnalysisResult(
+                verdict=verdict,
+                confidence=0.8,
+                statistical_evidence="test",
+                code_generated="print('test')",
+                execution_output="test",
+            )
+            assert result.verdict == verdict
+
+    def test_confidence_bounds(self) -> None:
+        """Confidence must be 0.0-1.0."""
+        with pytest.raises(ValueError):
+            AnalysisResult(
+                verdict="SUPPORTED",
+                confidence=1.5,  # Invalid
+                statistical_evidence="test",
+                code_generated="test",
+                execution_output="test",
+            )