feat(asr): add optional Nemotron voice-note input

AGENTS.md

@@ -225,6 +225,8 @@ hf spaces variables add build-small-hackathon/Fabella \
     --env MODAL_JUDGE_URL=https://khoitruong071510--fabella-serve-judge.modal.run
 hf spaces variables add build-small-hackathon/Fabella \
     --env MODAL_TTS_URL=https://khoitruong071510--fabella-serve-tts.modal.run
+hf spaces variables add build-small-hackathon/Fabella \
+    --env MODAL_ASR_URL=https://khoitruong071510--fabella-asr-experiment-serve-asr.modal.run
 
 # HF Space: upload code
 hf upload build-small-hackathon/Fabella app.py    --type space

README.md

@@ -73,18 +73,19 @@ The parent sees the validated draft, not a raw model output. If the judge reject
 | **Drafter** | `google/gemma-4-E4B-it` | 4B | Modal A10G · vLLM | Apache 2.0, fast on short empathetic text, native tool calling | **LangGraph ReAct** — needs the state machine (draft → validate → revise → end) with tool calls and middleware-driven early exit |
 | **Judge** | `nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16` | 4B | Modal A10G · vLLM | Follows structured-output instructions reliably | **Pydantic v2** + one LLM call + one repair retry — task is bounded, no agent loop needed |
 | **Read aloud** | `openbmb/VoxCPM2` | ~2B | Modal L4 · FastAPI | Apache 2.0, 48 kHz, voice-description control | Separate FastAPI server; only called when the user clicks **Read aloud** |
+| **Voice note** | `nvidia/nemotron-3.5-asr-streaming-0.6b` | 0.6B | Modal T4 · NeMo | Small multilingual streaming ASR with language prompts | Optional **Record** button: transcribes a short parent voice note into the textbox for review before drafting |
 
 The split is deliberate. The drafter needs agentic machinery (state machine, tool calls, conditional edges, jump-to-end). The judge doesn't — its job is "receive rubric + draft, return a structured verdict." Pydantic gives disciplined output, type safety, and a one-shot repair retry. Two layers, two files, two execution models: `agent.py` for the loop, `judge.py` for the verdict.
 
-All three models sit comfortably under the **32B cap** — Fabella uses **10B of parameters total** for inference, with the largest single model at 4B. That makes Fabella a candidate for the **Tiny Titan** special award (≤4B).
+The core drafter/judge/read-aloud path uses **10B of parameters total**, with the largest single model at 4B. The optional voice-note input adds a separate 0.6B ASR model. The largest single model remains 4B, so Fabella is still a candidate for the **Tiny Titan** special award (≤4B).
 
 ---
 
 ## Sponsor prize notes
 
 - **OpenAI / Codex** — Codex was used as a coding assistant for early boilerplate and scaffolding. This sponsor-track note is about development assistance, not runtime inference: Fabella's model pipeline uses Gemma, Nemotron, and VoxCPM2.
-- **NVIDIA** — `nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16` is the second model in the pipeline and acts as the structured-output judge in `judge.py`.
-- **Modal** — Modal runs all three inference services: the Gemma drafter, the Nemotron judge, and the VoxCPM2 TTS service.
+- **NVIDIA** — `nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16` is the second model in the pipeline and acts as the structured-output judge in `judge.py`. `nvidia/nemotron-3.5-asr-streaming-0.6b` powers the optional **Record** voice-note input.
+- **Modal** — Modal runs the inference services: the Gemma drafter, the Nemotron judge, the VoxCPM2 TTS service, and the isolated T4 ASR experiment endpoint.
 - **OpenBMB** — `openbmb/VoxCPM2` powers the optional **Read aloud** feature.
 
 ---
@@ -94,7 +95,8 @@ All three models sit comfortably under the **32B cap** — Fabella uses **10B of
 - **HF Space (CPU)** — custom HTML + CSS + JS frontend served by `gradio.Server` (FastAPI subclass). Chat-style, parent-friendly UI: welcome screen with example situations, alternating parent / Fabella turns, per-turn Read-aloud button, no default Gradio chrome.
 - **HF OAuth** — enabled for personalization; unsigned users fall back to browser-local anonymous sessions.
 - **HF Bucket per-user JSON** — minimal chat history and parent preferences persist at `/data/fabella-data/user-<owner_key>.json` (signed-in users keyed by HF username, anonymous users keyed by a `localStorage` session ID).
-- **Modal** — one app, three web servers, all `min_containers=0` with a 2-minute `scaledown_window` so they cold-start on demand (3-day demo budget):
+- **Voice note ASR** — the optional **Record** button uses the browser `MediaRecorder`, sends a short base64 audio note to the Space's `transcribe_audio` API, and the Space proxies it to an isolated Modal T4 endpoint (`modal_asr_app.py`). The ASR endpoint follows NVIDIA NeMo's documented cache-aware streaming path (`set_inference_prompt`, `CacheAwareStreamingAudioBuffer`, `conformer_stream_step`) rather than plain `transcribe()`.
+- **Modal core app** — one app, three web servers, all `min_containers=0` with a 2-minute `scaledown_window` so they cold-start on demand (3-day demo budget):
   - **Drafter** (A10G) — vLLM with `--language-model-only --enable-auto-tool-choice --tool-call-parser gemma4 --enforce-eager --safetensors-load-strategy eager --max-model-len 8192`
   - **Judge** (A10G) — vLLM with `--enforce-eager --safetensors-load-strategy eager --max-model-len 4096` (no tool-calling flags; Nemotron's tool-call dialect isn't a vLLM built-in)
   - **TTS** (L4) — VoxCPM2 wrapped in a tiny FastAPI app on the smallest GPU that fits
@@ -102,6 +104,7 @@ All three models sit comfortably under the **32B cap** — Fabella uses **10B of
   - **Aggressive summarization in `agent.py`**: `_build_user_prompt` keeps the last 2 conversation turns verbatim and compresses everything older into a single short line capped at 320 chars. This is what lets us run the drafter at `--max-model-len 8192` instead of the model's nominal 32k, and it directly reduces per-request drafter token cost on long follow-up conversations.
   - **Cold-start tunings**: `--enforce-eager` skips CUDA-graph capture (saves 20–40s of cold start at a small per-token throughput cost). `VLLM_DEEP_GEMM_WARMUP=skip` skips the dense-model MoE kernel warmup. `VLLM_USE_AOT_COMPILE=1` + `VLLM_CACHE_ROOT=/root/.cache/vllm` lets torch.compile artifacts persist across cold starts via the cache volume.
   - **No warmup ping on Space import.** The previous deployment fired a `/health` request to each endpoint on Space startup so the first parent click would land on a warm container. We removed it: every Space restart (code push, env-var change, periodic rebalance) paid for an A10G cold start whether or not a parent ever arrived. With the ping gone, the first request after a quiet period still pays a 30-60s cold start (image-baked weights, eager mode, AOT compile cache, deep-gemm warmup skip) and the 2-minute `scaledown_window` keeps a parent who reads the welcome screen and clicks a chip on a warm container for free.
+- **Modal ASR experiment app** — separate `fabella-asr-experiment` deployment on T4 with `min_containers=0`; only wakes when a parent clicks **Record**.
 - **LangChain 1.x** ReAct loop with a custom middleware (`FabellaAgentMiddleware`) that jumps to `end` after a successful validation or after a hard cap of two tool calls. The `@hook_config(can_jump_to=["end"])` is required — without it the early-exit silently does nothing.
 - **Pydantic v2** for the judge's structured output. `JudgeVerdict` has five fields (`ok`, `issues`, `score`, `verdict`, `reasoning`); cross-field consistency (`ok` ⇔ `verdict`) is enforced in code, not in the prompt.
 

app.py

@@ -78,6 +78,10 @@ MODAL_TTS_URL = os.environ.get(
     "MODAL_TTS_URL",
     "https://khoitruong071510--fabella-serve-tts.modal.run",
 )
+MODAL_ASR_URL = os.environ.get(
+    "MODAL_ASR_URL",
+    "https://khoitruong071510--fabella-asr-experiment-serve-asr.modal.run",
+)
 
 # HF Spaces bucket mount. The current Space has a writable bucket mounted at
 # /models. We avoid a separate database cloud API by storing one minimal JSON
@@ -464,6 +468,61 @@ def make_audio(text: str, tone: str, opener: str = "", body: str = "", closer: s
     return _make_audio_sync(text, tone, opener, body, closer, followup)
 
 
+def _validate_audio_data_url(data_url: str) -> None:
+    if not data_url.startswith("data:audio/") or "," not in data_url:
+        raise ValueError("Expected a browser audio recording data URL.")
+    header, payload = data_url.split(",", 1)
+    if ";base64" not in header:
+        raise ValueError("Audio recording must be base64 encoded.")
+    # Validate and cap size before proxying. Browser MediaRecorder output is
+    # compact; 8 MB is plenty for a short parent voice note.
+    audio = base64.b64decode(payload, validate=True)
+    if len(audio) > 8 * 1024 * 1024:
+        raise ValueError("Voice note is too large. Please keep it under about 30 seconds.")
+    if len(audio) < 256:
+        raise ValueError("Voice note was empty.")
+
+
+def _transcribe_audio_sync(data_url: str, target_lang: str = "en-US") -> str:
+    clean_lang = (target_lang or "en-US").strip()
+    if clean_lang != "auto" and not re.match(r"^[a-z]{2}-[A-Z]{2}$", clean_lang):
+        clean_lang = "en-US"
+    try:
+        _validate_audio_data_url(data_url)
+    except Exception as e:
+        return f"ERROR: {e}"
+    payload = json.dumps({"audio_data_url": data_url, "target_lang": clean_lang}).encode("utf-8")
+    req = urllib.request.Request(
+        MODAL_ASR_URL.rstrip("/") + "/transcribe",
+        data=payload,
+        headers={"Content-Type": "application/json", "Accept": "application/json"},
+        method="POST",
+    )
+    try:
+        with urllib.request.urlopen(req, timeout=600) as res:
+            body = json.loads(res.read().decode("utf-8"))
+    except urllib.error.HTTPError as e:
+        detail = e.read().decode("utf-8", errors="replace")[:300]
+        print(f"[asr] modal HTTP error: {e.code}: {detail}", flush=True)
+        return f"ERROR: Voice transcription failed: HTTP {e.code}"
+    except Exception as e:
+        print(f"[asr] modal error: {type(e).__name__}: {e}", flush=True)
+        return f"ERROR: Voice transcription failed: {type(e).__name__}: {e}"
+    if not body.get("ok"):
+        return f"ERROR: Voice transcription failed: {body.get('error') or 'unknown error'}"
+    text = str(body.get("text") or "").strip()
+    return text or "ERROR: Nemotron ASR returned an empty transcript."
+
+
+@app.api(name="transcribe_audio")
+def transcribe_audio(data_url: str, target_lang: str = "en-US") -> str:
+    """Transcribe a short browser-recorded voice note via Modal T4 ASR.
+
+    The ASR runtime is deliberately isolated in ``modal_asr_app.py`` so NeMo,
+    ffmpeg, and CUDA dependencies cannot break the HF Space build.
+    """
+    return _transcribe_audio_sync(data_url, target_lang)
+
 
 @app.get("/api/me")
 async def api_me(request: Request):
@@ -929,6 +988,19 @@ a { color: var(--accent-strong); }
 }
 .composer textarea::placeholder { color: var(--text-muted); }
 .composer-controls { display: flex; align-items: center; gap: 8px; }
+.btn-record {
+  border: 1px solid var(--line);
+  background: var(--bg);
+  color: var(--text-soft);
+  border-radius: 999px;
+  padding: 10px 12px;
+  font: 600 13px var(--font-sans);
+  cursor: pointer;
+  white-space: nowrap;
+}
+.btn-record:hover:not([disabled]) { border-color: var(--accent); color: var(--accent-strong); }
+.btn-record.is-recording { border-color: var(--danger); color: var(--danger); background: color-mix(in srgb, var(--danger) 8%, var(--surface)); }
+.btn-record[disabled] { opacity: 0.55; cursor: not-allowed; }
 .btn-send {
   border: 0;
   background: var(--accent);
@@ -948,7 +1020,14 @@ a { color: var(--accent-strong); }
   text-align: center;
   margin-top: 8px;
 }
-
+.asr-status {
+  max-width: 760px;
+  margin: 6px auto 0;
+  font: 500 11px var(--font-mono);
+  color: var(--text-muted);
+  letter-spacing: 0.04em;
+  text-align: left;
+}
 .typing {
   display: inline-flex; align-items: center; gap: 6px;
   font: 500 13px var(--font-sans); color: var(--text-muted);
@@ -1002,10 +1081,12 @@ a { color: var(--accent-strong); }
   <form id="composer" class="composer-inner" novalidate>
     <textarea id="input" rows="1" placeholder="Describe the hard situation. A sentence or two is enough." maxlength="800" required></textarea>
     <div class="composer-controls">
+      <button type="button" class="btn-record" id="record-btn" title="Record a short voice note">Record</button>
       <button type="submit" class="btn-send" id="send-btn"><span id="send-label">Draft</span><span aria-hidden="true">&rarr;</span></button>
     </div>
   </form>
-  <div class="composer-hint">Fabella checks every draft against a six-criterion rubric. Read-aloud uses VoxCPM2 on demand.</div>
+  <div class="asr-status" id="asr-status"></div>
+  <div class="composer-hint">Fabella checks every draft against a six-criterion rubric. Record uses Nemotron 3.5 ASR on demand.</div>
 </section>
 
 <dialog id="settings" style="border:1px solid var(--line); border-radius: var(--radius); padding: 0; max-width: 480px; width: calc(100% - 32px); background: var(--surface); color: var(--text);">
@@ -1067,6 +1148,8 @@ a { color: var(--accent-strong); }
 
   var thread = document.getElementById("thread");
   var input = document.getElementById("input");
+  var recordBtn = document.getElementById("record-btn");
+  var asrStatus = document.getElementById("asr-status");
   var sendBtn = document.getElementById("send-btn");
   var sendLabel = document.getElementById("send-label");
   var ageChip = document.getElementById("age-chip");
@@ -1086,6 +1169,9 @@ a { color: var(--accent-strong); }
   var currentAge = 7;
   var currentTone = "gentle";
   var childNameValue = "";
+  var mediaRecorder = null;
+  var recordTimer = null;
+  var recordChunks = [];
 
   function escapeHTML(s) {
     return String(s).replace(/[&<>"']/g, function (c) { return ({"&":"&amp;","<":"&lt;",">":"&gt;",'"':"&quot;","'":"&#39;"})[c]; });
@@ -1270,9 +1356,86 @@ a { color: var(--accent-strong); }
   function setBusy(busy) {
     sendBtn.disabled = busy;
     input.disabled = busy;
+    if (recordBtn && !recordBtn.classList.contains("is-recording")) recordBtn.disabled = busy;
     sendLabel.textContent = busy ? "Drafting..." : "Draft";
   }
 
+  function setAsrStatus(text) {
+    if (asrStatus) asrStatus.textContent = text || "";
+  }
+
+  function blobToDataUrl(blob) {
+    return new Promise(function (resolve, reject) {
+      var reader = new FileReader();
+      reader.onload = function () { resolve(String(reader.result || "")); };
+      reader.onerror = function () { reject(reader.error || new Error("Could not read recording")); };
+      reader.readAsDataURL(blob);
+    });
+  }
+
+  async function transcribeBlob(blob) {
+    setAsrStatus("Transcribing with Nemotron 3.5 ASR. First run can take a minute...");
+    var dataUrl = await blobToDataUrl(blob);
+    var text = await readGradioString("transcribe_audio", [dataUrl, "en-US"]);
+    if (text.startsWith("ERROR:")) throw new Error(text.slice(6).trim());
+    input.value = text;
+    autosize();
+    input.focus();
+    setAsrStatus("Transcript inserted. Edit it before drafting if needed.");
+  }
+
+  async function startRecording() {
+    if (!navigator.mediaDevices || !window.MediaRecorder) {
+      setAsrStatus("This browser does not support voice recording.");
+      return;
+    }
+    try {
+      var stream = await navigator.mediaDevices.getUserMedia({ audio: true });
+      recordChunks = [];
+      var recorderOptions = MediaRecorder.isTypeSupported("audio/webm") ? { mimeType: "audio/webm" } : undefined;
+      mediaRecorder = new MediaRecorder(stream, recorderOptions);
+      mediaRecorder.ondataavailable = function (event) {
+        if (event.data && event.data.size > 0) recordChunks.push(event.data);
+      };
+      mediaRecorder.onstop = async function () {
+        stream.getTracks().forEach(function (track) { track.stop(); });
+        recordBtn.classList.remove("is-recording");
+        recordBtn.textContent = "Record";
+        recordBtn.disabled = true;
+        clearTimeout(recordTimer);
+        try {
+          var blob = new Blob(recordChunks, { type: mediaRecorder.mimeType || "audio/webm" });
+          await transcribeBlob(blob);
+        } catch (err) {
+          setAsrStatus(String(err.message || err));
+        } finally {
+          recordBtn.disabled = false;
+          mediaRecorder = null;
+        }
+      };
+      mediaRecorder.start();
+      recordBtn.classList.add("is-recording");
+      recordBtn.textContent = "Stop";
+      setAsrStatus("Recording. Keep it short; auto-stops at 30 seconds.");
+      recordTimer = setTimeout(function () {
+        if (mediaRecorder && mediaRecorder.state === "recording") mediaRecorder.stop();
+      }, 30000);
+    } catch (err) {
+      setAsrStatus("Microphone unavailable: " + String(err.message || err));
+    }
+  }
+
+  if (recordBtn) {
+    recordBtn.addEventListener("click", function () {
+      if (mediaRecorder && mediaRecorder.state === "recording") {
+        setAsrStatus("Preparing transcript...");
+        mediaRecorder.stop();
+        return;
+      }
+      startRecording();
+    });
+  }
+
   function _extractSseText(obj) {
     if (obj == null) return null;
     if (Array.isArray(obj)) return obj.find(function (v) { return typeof v === "string"; }) || null;

modal_app.py

@@ -5,7 +5,7 @@ judge) or L4 (TTS):
 
   serve_drafter (port 8000) — Gemma 4 E4B-IT (4B). Generates explanations.
   serve_judge   (port 8001) — Nemotron-3 Nano 4B. Scores the draft against
-                                the request and returns a structured verdict.
+                                 the request and returns a structured verdict.
   serve_tts     (port 8002) — VoxCPM2. Synthesizes read-aloud WAV audio.
 
 The judge runs after the drafter; if the verdict is "revise", the
@@ -38,11 +38,10 @@ The most effective mitigations, in order:
   2. **Skip CUDA-graph capture** with ``--enforce-eager`` for the demo.
      Drops cold start by ~20–40s. Trades a small amount of throughput
      for much faster first-token.
-  3. **Smaller judge** is not the lever here — the bottleneck is
-     vLLM's import + compile, not 4B vs 7B.
-  4. **Space-side warmup ping** on Space startup keeps the first parent
-     request warm (see ``app.py`` ``/health`` pattern). The cold
-     request still happens — just not in front of a parent.
+  3. **No Space-side warmup ping**. A warmup ping makes the first click
+     feel better, but every Space restart would pay for an A10G cold
+     start whether or not a parent ever arrives. For budget safety, only
+     a real parent action wakes Modal.
 
 Volume layout
 -------------
@@ -121,12 +120,13 @@ tts_image = (
 )
 
 
+
+
 # --- Model download (one entry per model) --------------------------------
 
 
-@app.function(image=download_image, volumes={MODEL_PATH: model_volume}, timeout=60 * 60)
-def download_drafter(force: bool = False):
-    """Pull Gemma 4 E4B-IT weights to the Volume (run once)."""
+def _download_drafter(force: bool = False):
+    """Pull Gemma 4 E4B-IT weights to the Volume/image layer."""
     from huggingface_hub import snapshot_download
     target = Path(MODEL_PATH) / DRAFTER_DIR
     if target.exists() and any(target.iterdir()) and not force:
@@ -148,8 +148,13 @@ def download_drafter(force: bool = False):
 
 
 @app.function(image=download_image, volumes={MODEL_PATH: model_volume}, timeout=60 * 60)
-def download_judge(force: bool = False):
-    """Pull Nemotron-Nano-9B-v2 weights to the Volume (run once)."""
+def download_drafter(force: bool = False):
+    """Pull Gemma 4 E4B-IT weights to the Volume (run once)."""
+    return _download_drafter(force=force)
+
+
+def _download_judge(force: bool = False):
+    """Pull Nemotron-Nano-4B weights to the Volume/image layer."""
     from huggingface_hub import snapshot_download
     target = Path(MODEL_PATH) / JUDGE_DIR
     if target.exists() and any(target.iterdir()) and not force:
@@ -170,6 +175,12 @@ def download_judge(force: bool = False):
     print("Judge download complete")
 
 
+@app.function(image=download_image, volumes={MODEL_PATH: model_volume}, timeout=60 * 60)
+def download_judge(force: bool = False):
+    """Pull Nemotron-Nano-4B weights to the Volume (run once)."""
+    return _download_judge(force=force)
+
+
 # --- vLLM servers --------------------------------------------------------
 
 MINUTES = 60
@@ -265,7 +276,7 @@ vllm_drafter_image = (
     vllm_image
     .env(VLLM_RUNTIME_ENV)
     .run_function(
-        download_drafter,
+        _download_drafter,
         volumes={MODEL_PATH: model_volume},
         force_build=False,
     )
@@ -277,7 +288,7 @@ vllm_judge_image = (
     vllm_image
     .env(VLLM_RUNTIME_ENV)
     .run_function(
-        download_judge,
+        _download_judge,
         volumes={MODEL_PATH: model_volume},
         force_build=False,
     )
@@ -428,9 +439,8 @@ async def synthesize(payload: dict):
 '''
 
 
-@app.function(image=download_image, volumes={MODEL_PATH: model_volume}, timeout=60 * 60)
-def download_tts(force: bool = False):
-    """Pull VoxCPM2 weights to the Volume (run once)."""
+def _download_tts(force: bool = False):
+    """Pull VoxCPM2 weights to the Volume/image layer."""
     from huggingface_hub import snapshot_download
     target = Path(MODEL_PATH) / TTS_DIR
     if target.exists() and any(target.iterdir()) and not force:
@@ -454,11 +464,17 @@ def download_tts(force: bool = False):
     print("TTS download complete")
 
 
+@app.function(image=download_image, volumes={MODEL_PATH: model_volume}, timeout=60 * 60)
+def download_tts(force: bool = False):
+    """Pull VoxCPM2 weights to the Volume (run once)."""
+    return _download_tts(force=force)
+
+
 # Bake VoxCPM2 weights into the TTS image so cold start only has to
 # load them to VRAM (~5-10s), not download from the Volume (~10-20s).
 # Defined after ``download_tts`` so the forward reference resolves.
 tts_image_baked = tts_image.run_function(
-    download_tts,
+    _download_tts,
     volumes={MODEL_PATH: model_volume},
     force_build=False,
 )

modal_asr_app.py

@@ -0,0 +1,294 @@
+"""Isolated Nemotron 3.5 ASR experiment on Modal T4.
+
+This is deliberately separate from ``modal_app.py`` so ASR can be tested
+without adding a fourth endpoint to the production Fabella app. Deploy/run this
+first, call ``/transcribe`` manually, then wire the HF Space only if it works.
+"""
+
+import base64
+import json
+import os
+import re
+import subprocess
+import tempfile
+from pathlib import Path
+
+import modal
+
+
+app = modal.App("fabella-asr-experiment")
+
+MODEL_PATH = "/models"
+ASR_REPO = "nvidia/nemotron-3.5-asr-streaming-0.6b"
+ASR_DIR = "nemotron-3.5-asr-streaming-0.6b"
+ASR_NEMO = "nemotron-3.5-asr-streaming-0.6b.nemo"
+ASR_PORT = 8003
+MINUTES = 60
+
+model_volume = modal.Volume.from_name("fabella-asr-models", create_if_missing=True)
+cache_volume = modal.Volume.from_name("fabella-asr-cache", create_if_missing=True)
+
+
+download_image = (
+    modal.Image.debian_slim(python_version="3.11")
+    .pip_install("huggingface_hub[hf_xet]>=0.24")
+    .env({"HF_HUB_CACHE": MODEL_PATH})
+)
+
+
+def _download_asr(force: bool = False):
+    from huggingface_hub import snapshot_download
+
+    target = Path(MODEL_PATH) / ASR_DIR
+    if target.exists() and (target / ASR_NEMO).exists() and not force:
+        print(f"ASR checkpoint already at {target}; skipping", flush=True)
+        return
+    print(f"Downloading {ASR_REPO} to {target}...", flush=True)
+    snapshot_download(
+        repo_id=ASR_REPO,
+        local_dir=str(target),
+        allow_patterns=[ASR_NEMO],
+    )
+    model_volume.commit()
+    print("ASR download complete", flush=True)
+
+
+@app.function(image=download_image, volumes={MODEL_PATH: model_volume}, timeout=60 * MINUTES)
+def download_asr(force: bool = False):
+    """Pull the .nemo checkpoint once before deploying the web endpoint."""
+    return _download_asr(force=force)
+
+
+asr_image = (
+    modal.Image.from_registry("nvidia/cuda:12.9.0-devel-ubuntu22.04", add_python="3.11")
+    .entrypoint([])
+    .apt_install("ffmpeg", "git", "libsndfile1")
+    .pip_install(
+        "Cython",
+        "packaging",
+        "torch>=2.5.0",
+        "torchaudio>=2.5.0",
+        "soundfile",
+        "fastapi>=0.110",
+        "uvicorn[standard]>=0.27",
+        "huggingface_hub[hf_xet]>=0.24",
+        "git+https://github.com/NVIDIA/NeMo.git@main#egg=nemo_toolkit[asr]",
+    )
+    .env(
+        {
+            "HF_HUB_CACHE": MODEL_PATH,
+            "TORCH_HOME": "/cache/torch",
+            "NUMBA_CACHE_DIR": "/cache/numba",
+        }
+    )
+    .run_function(_download_asr, volumes={MODEL_PATH: model_volume}, force_build=False)
+)
+
+
+ASR_SERVER_PY = r'''
+import base64
+import json
+import os
+import re
+import subprocess
+import tempfile
+import traceback
+from pathlib import Path
+from threading import Lock
+
+from fastapi import FastAPI, HTTPException
+
+os.environ.setdefault("HF_HUB_CACHE", "/models")
+os.environ.setdefault("TORCH_HOME", "/cache/torch")
+os.environ.setdefault("NUMBA_CACHE_DIR", "/cache/numba")
+
+MODEL_PATH = "/models/nemotron-3.5-asr-streaming-0.6b/nemotron-3.5-asr-streaming-0.6b.nemo"
+
+print("[asr] importing torch/nemo", flush=True)
+import torch
+import nemo.collections.asr as nemo_asr
+from nemo.collections.asr.parts.utils.rnnt_utils import Hypothesis
+from nemo.collections.asr.parts.utils.streaming_utils import CacheAwareStreamingAudioBuffer
+
+print(f"[asr] restoring {MODEL_PATH}", flush=True)
+_model = nemo_asr.models.ASRModel.restore_from(MODEL_PATH, map_location="cuda" if torch.cuda.is_available() else "cpu")
+_model.eval()
+if torch.cuda.is_available():
+    _model = _model.cuda()
+print(f"[asr] loaded on {'cuda' if torch.cuda.is_available() else 'cpu'}", flush=True)
+_model_lock = Lock()
+
+app = FastAPI()
+
+
+def _suffix(header: str) -> str:
+    header = header.lower()
+    if "wav" in header:
+        return ".wav"
+    if "mp4" in header or "m4a" in header:
+        return ".m4a"
+    if "ogg" in header:
+        return ".ogg"
+    return ".webm"
+
+
+def _decode_data_url(data_url: str) -> tuple[bytes, str]:
+    if not data_url.startswith("data:audio/") or "," not in data_url:
+        raise HTTPException(status_code=400, detail="expected audio data URL")
+    header, payload = data_url.split(",", 1)
+    if ";base64" not in header:
+        raise HTTPException(status_code=400, detail="expected base64 audio data URL")
+    audio = base64.b64decode(payload, validate=True)
+    if len(audio) > 8 * 1024 * 1024:
+        raise HTTPException(status_code=413, detail="audio too large")
+    if len(audio) < 256:
+        raise HTTPException(status_code=400, detail="audio was empty")
+    return audio, _suffix(header)
+
+
+def _normalize_result(result) -> str:
+    if isinstance(result, tuple):
+        result = result[0]
+    if isinstance(result, list):
+        result = result[0] if result else ""
+    text = getattr(result, "text", result)
+    if isinstance(text, list):
+        text = text[0] if text else ""
+    return re.sub(r"\s+<[^>]+>\s*$", "", str(text or "")).strip()
+
+
+def _extract_transcriptions(hyps):
+    if not hyps:
+        return []
+    if isinstance(hyps[0], Hypothesis):
+        return [hyp.text for hyp in hyps]
+    return hyps
+
+
+def _transcribe_wav(wav_path: str, target_lang: str) -> str:
+    # Follow NeMo's documented cache-aware streaming path for
+    # prompt-conditioned models. Plain ASRModel.transcribe() does not populate
+    # the per-stream prompt correctly for this checkpoint.
+    with _model_lock:
+        if hasattr(_model, "set_inference_prompt"):
+            _model.set_inference_prompt(target_lang)
+            if hasattr(_model, "decoding") and hasattr(_model.decoding, "set_strip_lang_tags"):
+                _model.decoding.set_strip_lang_tags(True)
+        if hasattr(_model.encoder, "set_default_att_context_size"):
+            _model.encoder.set_default_att_context_size(att_context_size=[56, 13])
+
+        device = next(_model.parameters()).device
+        compute_dtype = torch.float32
+        streaming_buffer = CacheAwareStreamingAudioBuffer(
+            model=_model,
+            online_normalization=False,
+            pad_and_drop_preencoded=False,
+        )
+        streaming_buffer.append_audio_file(wav_path, stream_id=-1)
+        batch_size = len(streaming_buffer.streams_length)
+        cache_last_channel, cache_last_time, cache_last_channel_len = _model.encoder.get_initial_cache_state(
+            batch_size=batch_size
+        )
+        previous_hypotheses = None
+        pred_out_stream = None
+        transcribed_texts = []
+        for step_num, (chunk_audio, chunk_lengths) in enumerate(iter(streaming_buffer)):
+            chunk_audio = chunk_audio.to(device=device, dtype=compute_dtype)
+            chunk_lengths = chunk_lengths.to(device=device)
+            (
+                pred_out_stream,
+                transcribed_texts,
+                cache_last_channel,
+                cache_last_time,
+                cache_last_channel_len,
+                previous_hypotheses,
+            ) = _model.conformer_stream_step(
+                processed_signal=chunk_audio,
+                processed_signal_length=chunk_lengths,
+                cache_last_channel=cache_last_channel,
+                cache_last_time=cache_last_time,
+                cache_last_channel_len=cache_last_channel_len,
+                keep_all_outputs=streaming_buffer.is_buffer_empty(),
+                previous_hypotheses=previous_hypotheses,
+                previous_pred_out=pred_out_stream,
+                drop_extra_pre_encoded=0 if step_num == 0 else _model.encoder.streaming_cfg.drop_extra_pre_encoded,
+                return_transcription=True,
+            )
+        texts = _extract_transcriptions(transcribed_texts)
+    return _normalize_result(texts[0] if texts else "")
+
+
+@app.get("/health")
+async def health():
+    return {"ok": True, "model": "nvidia/nemotron-3.5-asr-streaming-0.6b", "cuda": bool(torch.cuda.is_available())}
+
+
+@app.post("/transcribe")
+async def transcribe(payload: dict):
+    data_url = str(payload.get("audio_data_url") or "")
+    target_lang = str(payload.get("target_lang") or "auto").strip()
+    if target_lang != "auto" and not re.match(r"^[a-z]{2}-[A-Z]{2}$", target_lang):
+        target_lang = "auto"
+    audio, suffix = _decode_data_url(data_url)
+    try:
+        with tempfile.TemporaryDirectory(prefix="fabella-asr-") as tmp:
+            src = Path(tmp) / f"input{suffix}"
+            wav = Path(tmp) / "converted.wav"
+            src.write_bytes(audio)
+            subprocess.run(
+                ["ffmpeg", "-hide_banner", "-loglevel", "error", "-y", "-i", str(src), "-ac", "1", "-ar", "16000", str(wav)],
+                check=True,
+                stdout=subprocess.DEVNULL,
+                stderr=subprocess.PIPE,
+            )
+            text = _transcribe_wav(str(wav), target_lang)
+    except subprocess.CalledProcessError as e:
+        detail = (e.stderr or b"").decode("utf-8", errors="replace")[:300]
+        raise HTTPException(status_code=400, detail=f"ffmpeg failed: {detail}")
+    except Exception as e:
+        print(f"[asr] transcribe failed: {type(e).__name__}: {e}", flush=True)
+        traceback.print_exc()
+        raise HTTPException(status_code=500, detail=f"transcribe failed: {e}")
+    return {"ok": True, "text": text, "target_lang": target_lang}
+
+
+@app.post("/warmup")
+async def warmup():
+    return {"ok": True, "message": "model loaded"}
+'''
+
+
+@app.function(
+    image=asr_image,
+    gpu="T4",
+    min_containers=0,
+    scaledown_window=2 * MINUTES,
+    timeout=10 * MINUTES,
+    volumes={MODEL_PATH: model_volume, "/cache": cache_volume},
+)
+@modal.concurrent(max_inputs=2)
+@modal.web_server(port=ASR_PORT, startup_timeout=15 * MINUTES)
+def serve_asr():
+    server_path = "/root/asr_server.py"
+    with open(server_path, "w") as f:
+        f.write(ASR_SERVER_PY)
+    cmd = [
+        "uvicorn",
+        "asr_server:app",
+        "--app-dir",
+        "/root",
+        "--host",
+        "0.0.0.0",
+        "--port",
+        str(ASR_PORT),
+        "--log-level",
+        "info",
+    ]
+    print(f"Starting ASR experiment: {' '.join(cmd)}", flush=True)
+    subprocess.Popen(cmd)
+
+
+@app.local_entrypoint()
+def main():
+    print("Deploy with: .venv/bin/modal deploy modal_asr_app.py")
+    print("Download only: .venv/bin/modal run modal_asr_app.py::download_asr")