WUMBOLABS / EVALUATIONS CANONICAL MODEL EVALUATION

/evaluations/grug-12b

Grug 12B

Published evaluation evidence and testing history. Results remain attributed to their original configurations and methods.

latest evidence 2026-07-04

Overview

kai-os · 1 tested configuration · 2 evidence events · newest evidence 2026-07-04.

Contents: Findings · Event results · Tested configurations · Chronology · Technical provenance

Findings

Each finding is attributed to the event and tested configuration that measured it; exact values, failures and limitations are in the event results. A historical event's recorded status describes that event's own sections at its date and is not a verdict on newer sections.

  • Recommended configuration: llama.cpp Q4_K_M (current): event result.
  • Latest evidence: 2026-07-04: event result
  • 2026-07-04 · llama.cpp · RTX 5070: classification BENCHMARK_ONLY (LMX local speed). Event result
  • 2026-07-04 · llama.cpp · RTX 5070: classification SPECIALIZED_TEST / UNSCORED_SMOKE. Event result

Event results

Newest first. Each event is one independent testing/publication event with its full report linked once at the top of its section; scores, failures and limitations stay visible and raw section-provenance JSON stays collapsed.

2026-07-04: LMX speed run

Benchmark Only: LMX local speed · config llama.cpp · RTX 5070 · maturity: BENCHMARK_ONLY · status: BENCHMARK_ONLY (LMX local speed)

Configuration: llama.cpp Q4_K_M.

Full report (REPORT.md)

LMX local speed evidence: 69.24 tok/s out (llama.cpp Q4_K_M).

This event is a bounded public-safe summary derived from retained WumboLabs evidence. It is not a WELP characterization, a universal model ranking, or a production-readiness proof. Values are attributed to the tested artifact, runtime, hardware, suite, and settings.

Identity and scope
  • Profile: grug-12b-llamacpp-q4km: llama.cpp Q4_K_M
  • Evidence maturity: BENCHMARK_ONLY
  • Evidence scope: performance
  • Hardware: WumboJetsII (RTX 5070 12GB)

2026-07-04: Honesty smoke + provenance refresh

Specialized Test: honesty/provenance · config llama.cpp · RTX 5070 · maturity: SPECIALIZED_TEST · status: SPECIALIZED_TEST / UNSCORED_SMOKE

Configuration: llama.cpp Q4_K_M.

Full report (REPORT.md)

Honesty smoke (2026-07-04) and provenance-refresh re-run (2026-07-24) for the Grug practical record.

This event is a bounded public-safe summary derived from retained WumboLabs evidence. It is not a WELP characterization, a universal model ranking, or a production-readiness proof. Values are attributed to the tested artifact, runtime, hardware, suite, and settings.

Identity and scope
  • Profile: grug-12b-llamacpp-q4km: llama.cpp Q4_K_M
  • Evidence maturity: SPECIALIZED_TEST
  • Evidence scope: specialized
  • Hardware: WumboJetsII (RTX 5070 12GB)

Tested configurations

Exact profile identities, sampling conditions and provenance are in each configuration's details below.

ConfigurationArtifact / precisionRuntimeHardwareContext / reasoningEvents
llama.cpp · RTX 5070kai-os/Grug-12B-GGUF Q4_K_Mllama.cppRTX 5070not recorded2 (2026-07-04, 2026-07-04)
Configuration details: llama.cpp · RTX 5070
  • Full profile name: llama.cpp Q4_K_M.
  • Profile identity: grug-12b-llamacpp-q4km.
  • Status: current canonical/recommended tested surface.
  • Artifact (as recorded): kai-os/Grug-12B-GGUF Q4_K_M
  • Runtime (as recorded): llama.cpp.
  • Profile Metadata

Events on this configuration:

Chronology

Compact index; full results are in Event results.

DateEventConfigurationStatus
2026-07-04LMX speed runllama.cpp · RTX 5070BENCHMARK_ONLY (LMX local speed)
2026-07-04Honesty smoke + provenance refreshllama.cpp · RTX 5070SPECIALIZED_TEST / UNSCORED_SMOKE

Technical provenance

Publication, acceptance and source-pin provenance for each event, collapsed; the scientific content above never depends on it.

Model evidence tree (GitHub)

Legacy provenance

Shared comparison events

This model appears in shared multi-model comparisons. Each report is stored once in WumboLabs/evaluations/shared-events/; the model's measured entries remain attributed to the same historical event: