Meridian powers our AI agents as they write and refine kernels for NVIDIA GPUs and CUDA. Explore its recorded optimization results, including generation speed, first-token delay, and GPU allocation.
Our AI agents use Meridian to improve how models run on NVIDIA GPUs. Compare recorded baseline performance with Geodd’s optimized configurations.
+51.8% higher generation rate
Mean time to first token was 23.8% lower in the Geodd run.
Baseline: vLLM with FP8 dynamic on H200. Geodd precision, batch settings, workload lengths, and runtime versions were not recorded for this pair.
+102.4% higher generation rate
Mean time to first token increased slightly, from 866.4 to 888.1 ms.
The Geodd run is FP8 and is also labeled custom batching. Baseline precision and RTX Pro 6000 generation are FP8. Full workload and runtime settings are incomplete.
+61.4% higher generation rate
Mean time to first token was nearly unchanged, at 740.6 to 741.5 ms.
The Geodd run is BF16 at batch 16 on A100. The recorded entry says Baseline; its serving engine and precision are unspecified. Full workload and runtime settings are incomplete.
In the recorded H200 comparison, Qwen 2.5 7B reached 195.16 tokens per second, up from 128.57 with the vLLM baseline. Mean time to first token fell from 789.2 ms to 601.4 ms.
| Mean metric | vLLM baseline | Geodd AI optimization |
|---|---|---|
| Generation rate | 128.57 tokens/s | 195.16 tokens/s |
| Time to first token | 789.2 ms | 601.4 ms |
| Time taken | 3.9344 s | 2.6599 s |
Across four recorded versions, Trinity Mini’s mean generation rate increased by 87.6% from the first version to the final recorded version.
Our case studies will explain the workload, the bottleneck, and the changes produced by the AI agent and its LLM. Each study will connect that work to the measured result, including the effect on production performance where deployment has been documented.
A recorded 51.8% increase in generation rate and 23.8% reduction in time to first token.
A configuration labeled “custom batching” reached 66.66 tokens/s, compared with 32.94 tokens/s for the vLLM baseline. Mean time to first token increased slightly, from 866.4 to 888.1 ms.
At batch 16, the Geodd BF16 run reached 33.49 tokens/s against a recorded baseline of 20.75 tokens/s. Mean time to first token was nearly unchanged.
These summaries show measured outcomes. Full engineering case studies will include the agent’s changes, verification, and production results where documented. Successful kernels and measured results inform the next experiments and further development of our LLMs.
Model-specific runtime kernels developed by our Meridian-powered AI agents are packaged with vLLM, SGLang, or TensorRT-LLM in free, public Docker images for the community. Use these runtimes in your own projects and evaluate our claims. We’ll keep adding to the runtime catalog.
Performance depends on your hardware and workload. A public image tag alone does not guarantee reproduction of a recorded benchmark.