/evaluations/qwen2.5-3b
Qwen2.5-3B Instruct
Published evaluation evidence and testing history. Results remain attributed to their original configurations and methods.
latest evidence 2026-07-15
Overview
Alibaba (Qwen team) · 2 tested configurations · 2 evidence events · newest evidence 2026-07-15.
Contents: Findings · Event results · Tested configurations · Chronology · Technical provenance
Findings
Each finding is attributed to the event and tested configuration that measured it; exact values, failures and limitations are in the event results. A historical event's recorded status describes that event's own sections at its date and is not a verdict on newer sections.
- Recommended configuration: vLLM BF16 (cross-runtime lane) (current): event result.
- Latest evidence: 2026-07-15: event result
- 2026-07-15 · vLLM · RTX 5070: classification SPECIALIZED_TEST / RUNTIME_FINGERPRINT. Event result
- 2026-07-15 · llama.cpp · RTX 5070: classification BENCHMARK_ONLY / CROSS_RUNTIME_METHODOLOGY_LANE. Event result
Event results
Newest first. Each event is one independent testing/publication event with its full report linked once at the top of its section; scores, failures and limitations stay visible and raw section-provenance JSON stays collapsed.
2026-07-15: vLLM fingerprint live smoke
Specialized Test: runtime fingerprint · config vLLM · RTX 5070 · maturity: SPECIALIZED_TEST · status: SPECIALIZED_TEST / RUNTIME_FINGERPRINT
Configuration: vLLM BF16 (cross-runtime lane).
vLLM runtime-fingerprint smoke using Qwen2.5-3B as payload.
This event is a bounded public-safe summary derived from retained WumboLabs evidence. It is not a WELP characterization, a universal model ranking, or a production-readiness proof. Values are attributed to the tested artifact, runtime, hardware, suite, and settings.
Identity and scope
- Profile:
qwen2.5-3b-vllm-bf16: vLLM BF16 (cross-runtime lane) - Evidence maturity: SPECIALIZED_TEST
- Evidence scope: specialized
- Hardware: WumboJetsII (RTX 5070 12GB)
2026-07-15: Cross-runtime comparison: llama.cpp F16 vs vLLM BF16
Benchmark Only: cross-runtime methodology lane · config llama.cpp · RTX 5070 · maturity: BENCHMARK_ONLY · status: BENCHMARK_ONLY / CROSS_RUNTIME_METHODOLOGY_LANE
Configuration: llama.cpp F16 (comparison reference).
LLMGauge cross-runtime methodology study with model-specific results: Qwen2.5-3B Instruct as F16 llama.cpp vs BF16 vLLM agent-backend comparison incl. failed-command-recovery probes. Primary subject is cross-runtime methodology; the Qwen2.5-3B results are retained as its benchmark-only evidence. Archaeology separately cites a full Qwen2.5-3B quant-lab record set (F16/Q8_0/Q6_K/Q5_K_M/Q4_K_M/IQ4_XS) from the Arch era.
This event is a bounded public-safe summary derived from retained WumboLabs evidence. It is not a WELP characterization, a universal model ranking, or a production-readiness proof. Values are attributed to the tested artifact, runtime, hardware, suite, and settings.
Identity and scope
- Profile:
qwen2.5-3b-llamacpp-f16: llama.cpp F16 (comparison reference) - Evidence maturity: BENCHMARK_ONLY
- Evidence scope: performance, agent-backend
- Hardware: WumboJetsII (RTX 5070 12GB)
Tested configurations
Exact profile identities, sampling conditions and provenance are in each configuration's details below.
| Configuration | Artifact / precision | Runtime | Hardware | Context / reasoning | Events |
|---|---|---|---|---|---|
| llama.cpp · RTX 5070 | Qwen2.5-3B-Instruct F16 GGUF (6,178,317,312-byte fingerprint) | llama.cpp | RTX 5070 | not recorded | 1 (2026-07-15) |
| vLLM · RTX 5070 | Qwen/Qwen2.5-3B-Instruct BF16 | vLLM | RTX 5070 | not recorded | 1 (2026-07-15) |
Configuration details: llama.cpp · RTX 5070
- Full profile name: llama.cpp F16 (comparison reference).
- Profile identity:
qwen2.5-3b-llamacpp-f16. - Status: current tested surface; not the recommended profile.
- Artifact (as recorded): Qwen2.5-3B-Instruct F16 GGUF (6,178,317,312-byte fingerprint)
- Runtime (as recorded): llama.cpp.
- Profile Metadata
Events on this configuration:
- Cross-runtime comparison: llama.cpp F16 vs vLLM BF16 (2026-07-15); status: BENCHMARK_ONLY / CROSS_RUNTIME_METHODOLOGY_LANE
Configuration details: vLLM · RTX 5070
- Full profile name: vLLM BF16 (cross-runtime lane).
- Profile identity:
qwen2.5-3b-vllm-bf16. - Status: current canonical/recommended tested surface.
- Artifact (as recorded): Qwen/Qwen2.5-3B-Instruct BF16
- Runtime (as recorded): vLLM.
- Profile Metadata
Events on this configuration:
- vLLM fingerprint live smoke (2026-07-15); status: SPECIALIZED_TEST / RUNTIME_FINGERPRINT
Chronology
Compact index; full results are in Event results.
| Date | Event | Configuration | Status |
|---|---|---|---|
| 2026-07-15 | vLLM fingerprint live smoke | vLLM · RTX 5070 | SPECIALIZED_TEST / RUNTIME_FINGERPRINT |
| 2026-07-15 | Cross-runtime comparison: llama.cpp F16 vs vLLM BF16 | llama.cpp · RTX 5070 | BENCHMARK_ONLY / CROSS_RUNTIME_METHODOLOGY_LANE |
Technical provenance
Publication, acceptance and source-pin provenance for each event, collapsed; the scientific content above never depends on it.