Mosaic powers our agents as they write and refine kernels for NVIDIA GPUs and CUDA. Explore the recorded results from Geodd’s NVIDIA optimization work, including generation speed, first-token delay, and GPU allocation.
Our AI agents use Geodd’s hardware-specific LLMs to improve how models run. Compare recorded baseline performance with Geodd’s optimized configurations.
+51.8% higher generation rate
Mean time to first token was 23.8% lower in the Geodd run.
Baseline: vLLM with FP8 dynamic on H200. Geodd precision, batch settings, workload lengths, and runtime versions were not recorded for this pair.
+102.4% higher generation rate
Mean time to first token increased slightly, from 866.4 to 888.1 ms.
The Geodd run is FP8 and is also labeled custom batching. Baseline precision and RTX Pro 6000 generation are unspecified. Full workload and runtime settings are incomplete.
+61.4% higher generation rate
Mean time to first token was nearly unchanged, at 740.6 to 741.5 ms.
The Geodd run is BF16 at batch 16 on A100. The recorded entry says Baseline; its serving engine and precision are unspecified. Full workload and runtime settings are incomplete.
Selected results from Geodd’s benchmark notes. Qwen 2.5 and Mistral Nemo use vLLM baselines. Qwen3 uses the recorded entry labeled “Baseline”; its serving engine is unspecified. These are configuration comparisons, with incomplete workload and runtime settings.
These comparisons do not isolate the contribution of a particular kernel change. Generation rate and time to first token are separate metrics.
In the recorded H200 comparison, Qwen 2.5 7B reached 195.16 tokens per second, up from 128.57 with the vLLM baseline. Mean time to first token fell from 789.2 ms to 601.4 ms.
| Mean metric | vLLM baseline | Geodd AI optimization |
|---|---|---|
| Generation rate | 128.57 tokens/s | 195.16 tokens/s |
| Time to first token | 789.2 ms | 601.4 ms |
| Time taken | 3.9344 s | 2.6599 s |
Baseline: vLLM with FP8 dynamic on H200. Geodd precision, batch settings, workload lengths, and runtime versions were not recorded for this pair. Results describe these runs; they do not isolate the contribution of a particular kernel change.
Across four recorded versions, Trinity Mini’s mean generation rate increased by 87.6% from the first version to the final recorded version.
This measures progress from our first recorded optimization version. Against the separately recorded vLLM baseline of 78.00 tokens/s, the final run’s generation rate was 46.8% higher. Its mean time to first token was also higher: 2.9265 s, compared with 0.7055 s for vLLM.
Our case studies will explain the workload, the bottleneck, and the changes produced by the AI agent and its LLM. Each study will connect that work to the measured result, including the effect on production performance where deployment has been documented.
A recorded 51.8% increase in generation rate and 23.8% reduction in time to first token.
A configuration labeled “custom batching” reached 66.66 tokens/s, compared with 32.94 tokens/s for the vLLM baseline. Mean time to first token increased slightly, from 866.4 to 888.1 ms.
At batch 16, the Geodd BF16 run reached 33.49 tokens/s against a recorded baseline of 20.75 tokens/s. Mean time to first token was nearly unchanged.
These summaries show measured outcomes. Full engineering case studies will include the agent’s changes, verification, and production results where documented. Successful kernels and measured results inform the next experiments and further development of our LLMs.
Selected optimized legacy inference models are available as free, public Docker images for the community. Use them in your own projects and evaluate our claims. We’ll keep adding to the local model catalog.
Choose a supported model and hardware configuration from the catalog. A public image tag alone does not guarantee reproduction of a recorded benchmark.