latency

Abstract illustration of vLLM prefix-cache head-of-line blocking — 11-second time-to-first-token gradient bar with stalled short requests stuck behind heavy prefix-caching requests; GPU with idle SMs on one side, active SMs on the other

eBPF, GPU Debugging, GPU Observability, MLOps

11-Second Time to First Token on a Healthy vLLM Server

Ingero Team / April 21, 2026

A healthy vLLM server, normal nvidia-smi output, 11 seconds to first token. eBPF uprobes on the CUDA driver and kernel tracepoints on the scheduler traced it to prefix caching head-of-line blocking.

GPU stream lane with active teal segments interrupted by red vertical sync bars at each tensor.cpu().numpy() call, demonstrating the 12 ms blocking stream sync per call

eBPF, GPU Debugging, GPU Observability, MLOps

Tracing a 13x PyTorch Slowdown to a Hidden NumPy Synchronization

Ingero Team / March 31, 2026

A PyTorch training loop, 13x slower than expected. torch.profiler showed nothing unusual. eBPF kernel tracing found a hidden synchronization point: NumPy triggering implicit CUDA sync on every batch.