01 / CONTROL
Match the setup
Both sides use the same model, H100 class, inputs, prompt, and output limit.
Vision methods
Do less work per frame.
Pandan removes repeated media work, tunes GPUs under load, and reuses visual context across a live stream.
ModelsQwen3-VL + Gemma 4
HardwareOne H100 each
Workload20 × 640×480 frames
QualityMatching test outputs
Our method
A fair test, end to end.
We change one layer at a time, then run Pandan and stock vLLM under the same conditions.
02 / LOAD
Measure real traffic
We track first-token latency, total latency, queue time, and throughput as concurrency rises.
03 / QUALITY
Check the answer
Performance gains stay experimental until paired tests show matching outputs.
What changed
Less preprocessing. Same model.
Pandan bypasses generic work only for inputs it can validate. Everything else returns to stock vLLM.
Qwen3-VL 30B-A3B
Reuse media work
Validate supported images and video, reuse cached items, and build tensors without rerunning the full processor.
−59%preprocessing time
−26%first-token latency
15/15matching outputs
Gemma 4 31B
Batch frame preparation
Batch independent frame work while preserving timestamps, positions, and prompt behavior. Unsupported inputs use stock processing.
−38%preprocessing time
−5%first-token latency
15/15matching outputs
BF16 preprocessing results. Tuned FP8 serving reaches 4.11 requests per second for Qwen and 0.78 for Gemma at 32 concurrent requests. Results are specific to this workload.
Built for streams
Stop starting over.
Reference stored frames instead of resending them with every request.
Pay mainly for what changed between questions.
Tie every result to its model, hardware, precision, load, and cache state.