Pandan Labs

Pandan API · video + voice · private beta

See it. Hear it. Respond.

Ask questions about video, synthesize streaming speech, or transcribe live audio. Pandan is building one low-latency media stack while labeling the release stage of every surface clearly.

https://pandan--pandan-api-api.modal.run

The live video and voice sections below call Pandan runtimes through server-side relays. Neither is a simulated request builder.

OpenAI compatibleVideo memoryStreaming PCMRealtime transcription

Product surface

One stack. Honest boundaries.

The badges below are release stages, not marketing labels. Customer video requests use the bearer API documented below. The embedded video and voice playgrounds use same-origin relays so provider credentials never enter the browser.

VisionLive beta

Video understanding

Ask once from an HTTPS URL or upload media and reuse a stable ID across follow-up questions.

Voice outputPrivate preview

Speech generation

OpenAI-compatible complete audio plus a WebSocket stream producing verified 24 kHz mono PCM.

Voice inputInstrumented

Streaming transcription

Binary 16 kHz PCM input with revisable partial transcripts, explicit finalization, and per-decode timing.

ConversationLive research

Full-duplex voice

Talk to the same NVIDIA VoiceChat runtime used on Pandan's voice page, including live audio, transcripts, and interruption handling.

Live video

Upload a video. Ask the real model.

This is Pandan's API-backed video playground, not a generated example. Your browser samples the video locally, then Pandan's server relay uploads those frames and streams answers from the optimized runtime.

Live · Pandan video runtimeOpen full video demo ↗
01 · BrowserSamples the selected video locally and prepares bounded temporal frames.
02 · Pandan relayValidates the request, uploads media, and keeps provider credentials server-side.
03 · Optimized runtimeRuns media-backed chat completions and streams the answer with timing metadata.
Real runtime, explicit GPU start.

Opening the docs and preparing frames do not wake a GPU. The playground starts one optimized H100 runtime only after you press Start. It uses the same media-backed chat protocol as the customer API through a protected demo relay; customer applications call /v1/media and /v1/chat/completions with their own project key.

Pricing

Transparent while we prove it.

The current video endpoint and approved voice evaluations are invited previews. They have protected access, but no self-serve billing gateway yet.

Private beta · current
$0 API usage

Build and benchmark video plus approved voice previews before production billing begins.

  • No credit card required
  • Video and approved voice evaluations
  • Per-key and per-window usage limits
  • Best-effort availability
Production · before GA
Usage-based

Published rates, spend controls, and service tiers will arrive before any customer is billed.

  • Model input and output rates
  • Visual processing disclosed separately
  • Usage dashboard and budgets
  • Volume and committed-use options
Beta access is invite-only and has no availability SLA. Video and voice credentials are currently separate; limits are assigned per key or evaluation window. Request access before relying on any preview for customers.

Quickstart

Set two variables.

Request a beta key from Pandan, then keep it on your server. Never ship the key in browser JavaScript.

Shell
export PANDAN_API_BASE="https://pandan--pandan-api-api.modal.run"
export PANDAN_API_KEY="your-key"
List models
curl "$PANDAN_API_BASE/v1/models" \
  -H "Authorization: Bearer $PANDAN_API_KEY"

Choose a flow

Direct or reusable.

01

One-off question

Put an HTTPS video URL directly in a chat completion. Best when you will ask once and discard the context.

OpenAI shapeNo upload step
02

Reusable video

Upload bytes once, receive a content-addressed media ID, and reuse it across questions. Best for interviews, inspections, and review workflows.

DeduplicatedWarm prefix reuse

One-off video

Ask from a URL.

The video must be reachable by HTTPS. MP4 data URLs also work, but repeatedly sending base64 increases request size.

POST /v1/chat/completions
curl "$PANDAN_API_BASE/v1/chat/completions" \
  -H "Authorization: Bearer $PANDAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-VL-30B-A3B-Instruct",
    "messages": [{
      "role": "user",
      "content": [
        {
          "type": "video_url",
          "video_url": {"url": "https://example.com/video.mp4"}
        },
        {"type": "text", "text": "What happens in this video?"}
      ]
    }],
    "temperature": 0,
    "max_tokens": 128
  }'

Reusable video

Upload once. Keep the ID.

Uploads are content-addressed: identical bytes return the same ID. MP4, WebM, and QuickTime are accepted up to 256 MiB.

1 · Upload
curl "$PANDAN_API_BASE/v1/media" \
  -H "Authorization: Bearer $PANDAN_API_KEY" \
  -H "Content-Type: video/mp4" \
  --data-binary @video.mp4
Response
{
  "id": "media_<sha256>_mp4",
  "object": "media",
  "content_type": "video/mp4",
  "bytes": 1549315,
  "created": true
}
2 · Ask and follow up
curl "$PANDAN_API_BASE/v1/chat/completions" \
  -H "Authorization: Bearer $PANDAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-VL-30B-A3B-Instruct",
    "media_ids": ["media_<sha256>_mp4"],
    "messages": [{
      "role": "user",
      "content": "Where is the person sitting?"
    }],
    "temperature": 0,
    "max_tokens": 128
  }'

Python

Use the OpenAI client.

Standard chat fields work through the OpenAI SDK. Pass Pandan's `media_ids` extension through `extra_body`.

Python
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["PANDAN_API_KEY"],
    base_url="https://pandan--pandan-api-api.modal.run/v1",
)

response = client.chat.completions.create(
    model="Qwen/Qwen3-VL-30B-A3B-Instruct",
    messages=[{
        "role": "user",
        "content": "What changes after the door opens?",
    }],
    temperature=0,
    max_tokens=128,
    extra_body={"media_ids": ["media_<sha256>_mp4"]},
)

print(response.choices[0].message.content)

Voice APIs

Speak, listen, then converse.

This is the API-backed experience from Pandan's voice page, placed directly in the docs. It captures and plays live audio while a server-side relay keeps Modal credentials out of the browser.

Live · Pandan voice APIOpen full voice page ↗
01 · Browser24 kHz microphone capture, audio playout, partial transcripts, and barge-in.
02 · Pandan session APIOrigin checks, bounded frames, backpressure, and server-held upstream credentials.
03 · VoiceChatOne live full-duplex model session on the private GPU runtime.
Live demo, not yet a general customer WebSocket.

The hosted relay only accepts sessions from Pandan's own origin. Before exposing it as a customer API, we still need project authentication, quotas, tenant isolation, metering, and a stable public session contract.

Reference

Endpoint surface.

GET
/health

Returns 200 after the model server is ready.

Public · may wake the GPU
GET
/v1/models

Lists the currently served model in OpenAI format.

Bearer auth
POST
/v1/media

Stores image or video bytes and returns a content-addressed media ID.

Bearer auth · 256 MiB maximum
HEAD
/v1/media/{media_id}

Checks whether uploaded media exists without returning its bytes.

Bearer auth
POST
/v1/chat/completions

Runs one-off or media-ID-backed multimodal inference.

Bearer auth · streaming supported
WSS
/api/voice/realtime

Powers Pandan's live full-duplex browser experience with audio, transcript, interruption, and session events.

Live research preview · Pandan origin only
Stable HTTP errors, evolving quotas.

400 invalid request or media ID · 401 invalid key · 413 media too large · 415 unsupported media · 5xx startup or inference failure. Private-beta limits are key-specific; rate-limit headers are not yet a stable contract.

Runtime behavior

What “indexed” means here.

Content-addressed, not semantic search.

The upload endpoint durably stores and deduplicates the video. The first inference processes its visual content. Follow-up requests reuse the stable media prefix while the GPU remains warm. After scale-to-zero, the media still exists, but GPU processor and prefix caches must warm again.

489 msone-off engine prefill
18 mswarm media-ID prefill
16 mssecond follow-up prefill

Observed August 7, 2026 on one Modal H100 using the same 1.55 MB sample video and short deterministic answers. This is a scoped integration measurement, not a general video benchmark.