/evaluations/gemma-4-12b
Gemma 4 12B IT
Published evaluation evidence and testing history. Results remain attributed to their original configurations and methods.
latest evidence 2026-11-09
Overview
Google · 5 tested configurations · 7 evidence events · newest evidence 2026-11-09.
Contents: Findings · Event results · Limitations · Tested configurations · Chronology · Technical provenance
Findings
Each finding is attributed to the event and tested configuration that measured it; exact values, failures and limitations are in the event results. A historical event's recorded status describes that event's own sections at its date and is not a verdict on newer sections.
- Model classification: READY_WITH_GUARDRAILS - attributed to the 2026-09-12 event, configuration UD-Q4_K_XL · llama.cpp b9672 · RTX 5070.
- Recommended configuration: llama.cpp official QAT Q4_0 (google GGUF, UD-Q4_K_XL packaging) (current): event result.
- Practical context: 32,768 default / 131,072 guarded tokens; native model-card maximum 262,144 (envelope complete: YES); source: event result
- Context limit outcome: FIT_LIMIT: measured: one bounded admission attempt failed with CUDA OOM (compute buffer) and KV-slope accounting (17.4 KiB/token) proves the point cannot fit with required reserve on 12GB; nearest measured boundary 131,072; source: event result
- Latest evidence: 2026-11-09: event result
- 2026-11-09 · UNKNOWN · llama.cpp b11827 · UNKNOWN: classification NOT_ASSIGNED; See configuration-specific section findings; no composite model score; advanced-reasoning 3 units: 3 PASS (ADVANCE); baseline 61 units: 48 PASS, 11 FAIL, 2 NOT_EVALUABLE; coverage failures retained: context 5 pass / 2 fail; instruction following 13 pass / 2 fail; reasoning 4 pass / 1 fail; reliability 17 pass / 3 fail; structured output 1 pass / 1 fail; tool use 1 pass / 2 fail. Event result
- 2026-09-12 · UD-Q4_K_XL · llama.cpp b9672 · RTX 5070: classification READY_WITH_GUARDRAILS; 12/12 frozen mechanical quality screen (frozen scorer, temp 0). Event result
- 2026-07-04 · UD-Q4_K_XL · llama.cpp b9672 · RTX 5070: classification PRACTICAL_USE / SHARED_MULTI_MODEL_COMPARISON (canonical). Event result
- 2026-07-04 · llama.cpp · RTX 5070: classification BENCHMARK_ONLY (LMX local speed). Event result
- 2026-06-21 · UD-Q4_K_XL · llama.cpp b9672 · RTX 5070: classification PRACTICAL_USE / SHARED_MULTI_MODEL_COMPARISON (canonical). Event result
- 2026-06-21 · UD-Q4_K_XL · llama.cpp b9672 · RTX 5070: classification SPECIALIZED_TEST / UNSCORED_SMOKE. Event result
- 2026-06-16 · UD-Q4_K_XL · llama.cpp b9672 · RTX 5070: classification BENCHMARK_ONLY. Event result
Event results
Newest first. Each event is one independent testing/publication event with its full report linked once at the top of its section; scores, failures and limitations stay visible and raw section-provenance JSON stays collapsed.
2026-11-09: Gemma 4 12B IT fresh Model/Core baseline + advanced reasoning (RTX 5070 CUDA)
WELP v1.0 fresh second-family baseline and reasoning arm · config UNKNOWN · llama.cpp b11827 · UNKNOWN · maturity: CURRENT_WELP · status: NOT_ASSIGNED
Configuration: UD-Q4_K_XL (Unsloth Dynamic 2.0 GGUF): Unsloth UD-Q4_K_XL GGUF, llama.cpp CUDA on RTX 5070, thinking not configurable, fresh NVMe (SSD) canonical deployment.
WELP outcome
- Outcome: Finalized independent section findings
- Classification: NOT_ASSIGNED
- Methodology snapshot: model-baseline/2
Publication state: published; canonical evidence linked above
Section results
Each section retains its own profile, methodology, outcomes and manifest. Performance observations are not model-quality scores; null/UNKNOWN metrics remain unavailable.
advanced-reasoning (reasoning-advanced/1)
- Units: 3 recorded (3 PASS). Per-unit outcomes are retained in the section provenance below.
- Advancement: ADVANCE.
baseline (model-baseline/2)
- Units: 61 recorded (48 PASS, 11 FAIL, 2 NOT_EVALUABLE). Per-unit outcomes are retained in the section provenance below.
| Unit | Outcome | Completion / execution | Limits | Reason |
|---|---|---|---|---|
| Q02 | FAIL | COMPLETE / VALID | NONE | not recorded |
| context-family-a-v2-3072-seed-42 | NOT_EVALUABLE | INCOMPLETE / VALID | BUDGET | BUDGET_LIMITED |
| context-multidocument-v2-6144-seed-42 | FAIL | COMPLETE / VALID | NONE | not recorded |
| context-multidocument-v2-8192-seed-42 | FAIL | COMPLETE / VALID | NONE | not recorded |
| hall-fake-package | FAIL | COMPLETE / VALID | NONE | not recorded |
| linux-diagnosis-stale-config-bind-failure | NOT_EVALUABLE | INCOMPLETE / VALID | BUDGET | BUDGET_LIMITED |
| linux-diagnosis-startup-invalid-config | FAIL | COMPLETE / VALID | NONE | not recorded |
| multi-turn-correction-01 | FAIL | COMPLETE / VALID | NONE | not recorded |
| strict-extract-09 | FAIL | COMPLETE / VALID | NONE | not recorded |
| structured-semantics-01 | FAIL | COMPLETE / VALID | NONE | not recorded |
| tool-recovery-01 | FAIL | COMPLETE / VALID | NONE | not recorded |
| tool-selection-01 | FAIL | COMPLETE / VALID | NONE | not recorded |
| uncertainty-api | FAIL | COMPLETE / VALID | NONE | not recorded |
- Coverage status: PARTIALLY_COMPLETE.
- Coverage rule (as recorded): Every fixed applicable task identity; one result per identity, no duplicated credit.
- Coverage advancement: NOT_CALIBRATED.
| Category | Pass | Fail | Measured | Scored | Status |
|---|---|---|---|---|---|
| coding | 5 | 0 | 5 | 5 | COMPLETE |
| context | 5 | 2 | 8 | 7 | COMPLETE_WITH_DOCUMENTED_LIMITATIONS |
| instruction_following | 13 | 2 | 15 | 15 | COMPLETE |
| performance | 0 | 0 | 0 | 0 | PARTIALLY_COMPLETE |
| reasoning | 4 | 1 | 6 | 5 | COMPLETE_WITH_DOCUMENTED_LIMITATIONS |
| reliability | 17 | 3 | 20 | 20 | COMPLETE |
| structured_output | 1 | 1 | 2 | 2 | COMPLETE |
| tool_use | 1 | 2 | 3 | 3 | COMPLETE |
- Timed samples: perf-short: 3 samples (35-token prompts): generation 65.33828742748229-65.47861437274294 tok/s. perf-substantial: 3 samples (3,635-token prompts): generation 57.93941682904009-58.09057568621107 tok/s. Full per-sample timings are retained in the section provenance below.
Quality and capabilities
- Constrained result: See configuration-specific section findings; no composite model score.
Guardrails and limitations
- Model and Agentic findings are independent.
- Performance is deployment-attributed, never model-quality credit.
- Missing metrics remain unknown; boundary memory samples are not true peaks.
Reliability: not recorded
LocalMaxxing
- Status: BLOCKED
- Submission reference: not recorded
- verifiedRun: null (not claimed)
No submission or service-state claim is made by this export.
Tested condition: artifact UNKNOWN · llama.cpp b11827 · UNKNOWN · context 16,384 default · envelope complete: NO · reasoning NOT_CONFIGURABLE · capacity 16,384 tokens.
Context limit outcome: NOT_CLAIMED
Raw section findings and provenance (JSON: outcomes, hashes, manifest and sampling conditions)
{
"profile_conditions": [
{
"artifact_sha256": "90fd944d227e9d9b68e7e2c7d5b57b79d4c66ed521b0919fbbd932cf834f6f8e",
"context_capacity": 16384,
"framework": {
"methodology": "model-baseline/2",
"version": "welp/1.0.0-alpha-m2"
},
"profile_id": "gemma-4-12b-rtx5070-cuda-ssd-not-configurable-2026-11-09",
"reasoning": {
"arm": "NOT_CONFIGURABLE",
"controls": {}
},
"runtime": {
"backend": "CUDA",
"engine": "llama.cpp",
"version": "b11827-970b51ce7c9ba5fff0726c5e9994f1c2f0e4eedc"
},
"sampling": {
"min_p": 0.0,
"presence_penalty": 0.0,
"repeat_penalty": 1.0,
"seed": 42,
"temperature": 1.0,
"top_k": 64,
"top_p": 0.95
},
"template_sha256": "aa3185dfc65051046349995de1cfc2fbb275491b9a3832b71595f1ff5b31d61b"
}
],
"sections": [
{
"advancement": {
"decision": "ADVANCE"
},
"evidence_kind": "REAL_MODEL",
"manifest_sha256": "801c85c3fa4ea9069c55bdf00daf418b8b0e41717077793dc86a555a31ecb4af",
"methodology": "reasoning-advanced/1",
"profile_id": "gemma-4-12b-rtx5070-cuda-ssd-not-configurable-2026-11-09",
"profile_revision": "7daaea329557700cd01dc92792976360f91c175b9ba22893dd4f9466fb1c7567",
"report_sha256": "fef41101f71507a58ef295a596714785f3b3f6941179b0bf7ca2859460b46cee",
"section": "advanced-reasoning",
"units": [
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "advanced-constraints-01"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "advanced-evidence-01"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "advanced-revision-01"
}
]
},
{
"coverage": {
"advancement": "NOT_CALIBRATED",
"categories": {
"coding": {
"applicable": 5,
"budget_limited": 0,
"excluded": {},
"fail": 0,
"indeterminate": 0,
"measured": 5,
"minimum_measurement_coverage_met": true,
"minimum_scoring_coverage_met": true,
"missing": [],
"pass": 5,
"planned": 5,
"scored": 5,
"status": "COMPLETE",
"unscored": []
},
"context": {
"applicable": 8,
"budget_limited": 1,
"excluded": {},
"fail": 2,
"indeterminate": 0,
"measured": 8,
"minimum_measurement_coverage_met": true,
"minimum_scoring_coverage_met": false,
"missing": [],
"pass": 5,
"planned": 8,
"scored": 7,
"status": "COMPLETE_WITH_DOCUMENTED_LIMITATIONS",
"unscored": [
"context-family-a-v2-3072-seed-42"
]
},
"instruction_following": {
"applicable": 15,
"budget_limited": 0,
"excluded": {},
"fail": 2,
"indeterminate": 0,
"measured": 15,
"minimum_measurement_coverage_met": true,
"minimum_scoring_coverage_met": true,
"missing": [],
"pass": 13,
"planned": 15,
"scored": 15,
"status": "COMPLETE",
"unscored": []
},
"performance": {
"applicable": 2,
"budget_limited": 0,
"excluded": {},
"fail": 0,
"indeterminate": 0,
"measured": 0,
"measurement_gaps": {
"perf-short": [
"memory_consumption"
],
"perf-substantial": [
"memory_consumption"
]
},
"minimum_measurement_coverage_met": false,
"minimum_scoring_coverage_met": null,
"missing": [
"perf-short",
"perf-substantial"
],
"pass": 0,
"planned": 2,
"scored": 0,
"status": "PARTIALLY_COMPLETE",
"unscored": []
},
"reasoning": {
"applicable": 6,
"budget_limited": 1,
"excluded": {},
"fail": 1,
"indeterminate": 0,
"measured": 6,
"minimum_measurement_coverage_met": true,
"minimum_scoring_coverage_met": false,
"missing": [],
"pass": 4,
"planned": 6,
"scored": 5,
"status": "COMPLETE_WITH_DOCUMENTED_LIMITATIONS",
"unscored": [
"linux-diagnosis-stale-config-bind-failure"
]
},
"reliability": {
"applicable": 20,
"budget_limited": 0,
"excluded": {},
"fail": 3,
"indeterminate": 0,
"measured": 20,
"minimum_measurement_coverage_met": true,
"minimum_scoring_coverage_met": true,
"missing": [],
"pass": 17,
"planned": 20,
"scored": 20,
"status": "COMPLETE",
"unscored": []
},
"structured_output": {
"applicable": 2,
"budget_limited": 0,
"excluded": {},
"fail": 1,
"indeterminate": 0,
"measured": 2,
"minimum_measurement_coverage_met": true,
"minimum_scoring_coverage_met": true,
"missing": [],
"pass": 1,
"planned": 2,
"scored": 2,
"status": "COMPLETE",
"unscored": []
},
"tool_use": {
"applicable": 3,
"budget_limited": 0,
"excluded": {},
"fail": 2,
"indeterminate": 0,
"measured": 3,
"minimum_measurement_coverage_met": true,
"minimum_scoring_coverage_met": true,
"missing": [],
"pass": 1,
"planned": 3,
"scored": 3,
"status": "COMPLETE",
"unscored": []
}
},
"classification": null,
"rule": "Every fixed applicable task identity; one result per identity, no duplicated credit",
"status": "PARTIALLY_COMPLETE"
},
"evidence_kind": "REAL_MODEL",
"manifest_sha256": "2386196149a088cca723c9e5c04c7d5bfbbd6541e8d109f33352fefc6b99a164",
"memory": {
"sampled_host_available_min_bytes": "UNKNOWN",
"sampled_vram_used_max_bytes": "UNKNOWN"
},
"methodology": "model-baseline/2",
"performance_observations": [
{
"elapsed_seconds": 2.850185274000978,
"sample": 1,
"timings": {
"cache_state": {
"cache_n": 0,
"prompt_n": 35
},
"first_byte_seconds": 1.2426216849999037,
"latency_seconds": 2.850185274000978,
"native_timings": {
"cache_n": 0,
"predicted_ms": 1607.021,
"predicted_n": 106,
"predicted_per_second": 65.33828742748229,
"predicted_per_token_ms": 15.304961904761905,
"prompt_ms": 1237.989,
"prompt_n": 35,
"prompt_per_second": 28.27165669484947,
"prompt_per_token_ms": 35.371114285714285
},
"streaming": true,
"tokens_per_second": {
"generation": 65.33828742748229,
"prompt": 28.27165669484947
},
"ttft_seconds": 1.2962998779985355,
"usage": {
"completion_tokens": 106,
"prompt_tokens": 35,
"prompt_tokens_details": {
"cached_tokens": 0
},
"total_tokens": 141
},
"usage_source": "sse_usage_frame"
},
"unit_id": "perf-short"
},
{
"elapsed_seconds": 2.7234282290010015,
"sample": 2,
"timings": {
"cache_state": {
"cache_n": 0,
"prompt_n": 35
},
"first_byte_seconds": 1.1188276479952037,
"latency_seconds": 2.7234282290010015,
"native_timings": {
"cache_n": 0,
"predicted_ms": 1603.577,
"predicted_n": 106,
"predicted_per_second": 65.47861437274294,
"predicted_per_token_ms": 15.272161904761905,
"prompt_ms": 1115.058,
"prompt_n": 35,
"prompt_per_second": 31.388501764033798,
"prompt_per_token_ms": 31.8588
},
"streaming": true,
"tokens_per_second": {
"generation": 65.47861437274294,
"prompt": 31.388501764033798
},
"ttft_seconds": 1.1683740000007674,
"usage": {
"completion_tokens": 106,
"prompt_tokens": 35,
"prompt_tokens_details": {
"cached_tokens": 0
},
"total_tokens": 141
},
"usage_source": "sse_usage_frame"
},
"unit_id": "perf-short"
},
{
"elapsed_seconds": 2.7329273679933976,
"sample": 3,
"timings": {
"cache_state": {
"cache_n": 0,
"prompt_n": 35
},
"first_byte_seconds": 1.1271861589921173,
"latency_seconds": 2.7329273679933976,
"native_timings": {
"cache_n": 0,
"predicted_ms": 1604.564,
"predicted_n": 106,
"predicted_per_second": 65.43833714329874,
"predicted_per_token_ms": 15.281561904761906,
"prompt_ms": 1122.664,
"prompt_n": 35,
"prompt_per_second": 31.175846023387226,
"prompt_per_token_ms": 32.07611428571428
},
"streaming": true,
"tokens_per_second": {
"generation": 65.43833714329874,
"prompt": 31.175846023387226
},
"ttft_seconds": 1.1784431889973348,
"usage": {
"completion_tokens": 106,
"prompt_tokens": 35,
"prompt_tokens_details": {
"cached_tokens": 0
},
"total_tokens": 141
},
"usage_source": "sse_usage_frame"
},
"unit_id": "perf-short"
},
{
"elapsed_seconds": 6.49369424401084,
"sample": 1,
"timings": {
"cache_state": {
"cache_n": 0,
"prompt_n": 3635
},
"first_byte_seconds": 4.162014397006715,
"latency_seconds": 6.49369424401084,
"native_timings": {
"cache_n": 0,
"predicted_ms": 2330.02,
"predicted_n": 136,
"predicted_per_second": 57.93941682904009,
"predicted_per_token_ms": 17.25940740740741,
"prompt_ms": 4128.43,
"prompt_n": 3635,
"prompt_per_second": 880.4799887608607,
"prompt_per_token_ms": 1.1357441540577717
},
"streaming": true,
"tokens_per_second": {
"generation": 57.93941682904009,
"prompt": 880.4799887608607
},
"ttft_seconds": 4.220767614999204,
"usage": {
"completion_tokens": 136,
"prompt_tokens": 3635,
"prompt_tokens_details": {
"cached_tokens": 0
},
"total_tokens": 3771
},
"usage_source": "sse_usage_frame"
},
"unit_id": "perf-substantial"
},
{
"elapsed_seconds": 6.508392004005145,
"sample": 2,
"timings": {
"cache_state": {
"cache_n": 0,
"prompt_n": 3635
},
"first_byte_seconds": 4.179393672005972,
"latency_seconds": 6.508392004005145,
"native_timings": {
"cache_n": 0,
"predicted_ms": 2327.693,
"predicted_n": 136,
"predicted_per_second": 57.99733899616486,
"predicted_per_token_ms": 17.24217037037037,
"prompt_ms": 4141.863,
"prompt_n": 3635,
"prompt_per_second": 877.6243926947849,
"prompt_per_token_ms": 1.139439614855571
},
"streaming": true,
"tokens_per_second": {
"generation": 57.99733899616486,
"prompt": 877.6243926947849
},
"ttft_seconds": 4.235855052000261,
"usage": {
"completion_tokens": 136,
"prompt_tokens": 3635,
"prompt_tokens_details": {
"cached_tokens": 0
},
"total_tokens": 3771
},
"usage_source": "sse_usage_frame"
},
"unit_id": "perf-substantial"
},
{
"elapsed_seconds": 6.472986538996338,
"sample": 3,
"timings": {
"cache_state": {
"cache_n": 0,
"prompt_n": 3635
},
"first_byte_seconds": 4.146988735999912,
"latency_seconds": 6.472986538996338,
"native_timings": {
"cache_n": 0,
"predicted_ms": 2323.957,
"predicted_n": 136,
"predicted_per_second": 58.09057568621107,
"predicted_per_token_ms": 17.214496296296296,
"prompt_ms": 4130.314,
"prompt_n": 3635,
"prompt_per_second": 880.0783669231927,
"prompt_per_token_ms": 1.136262448418157
},
"streaming": true,
"tokens_per_second": {
"generation": 58.09057568621107,
"prompt": 880.0783669231927
},
"ttft_seconds": 4.203767286991933,
"usage": {
"completion_tokens": 136,
"prompt_tokens": 3635,
"prompt_tokens_details": {
"cached_tokens": 0
},
"total_tokens": 3771
},
"usage_source": "sse_usage_frame"
},
"unit_id": "perf-substantial"
}
],
"profile_id": "gemma-4-12b-rtx5070-cuda-ssd-not-configurable-2026-11-09",
"profile_revision": "7daaea329557700cd01dc92792976360f91c175b9ba22893dd4f9466fb1c7567",
"report_sha256": "0cfe7ffdd935668bed1209a9b6ca1a43805e5c205e4de1f6852d4217bbdaac41",
"section": "baseline",
"units": [
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "Q01"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "FAIL",
"reason": null,
"scoring": "SCORED_FAIL",
"unit_id": "Q02"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "Q03"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "Q04"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "Q05"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "Q06"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "Q07"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "Q08"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "Q09"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "Q10"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "Q11"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "Q12"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "coding-comprehension-01"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "coding-correction-01"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "coding-debug-01"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "coding-generation-01"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "context-family-a-v2-3072-seed-314159"
},
{
"completion": "INCOMPLETE",
"execution": "VALID",
"limits": "BUDGET",
"outcome": "NOT_EVALUABLE",
"reason": "BUDGET_LIMITED",
"scoring": "NOT_SCOREABLE",
"unit_id": "context-family-a-v2-3072-seed-42"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "context-family-a-v2-4096-seed-314159"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "context-family-a-v2-4096-seed-42"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "context-family-a-v2-8192-seed-314159"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "context-family-a-v2-8192-seed-42"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "FAIL",
"reason": null,
"scoring": "SCORED_FAIL",
"unit_id": "context-multidocument-v2-6144-seed-42"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "FAIL",
"reason": null,
"scoring": "SCORED_FAIL",
"unit_id": "context-multidocument-v2-8192-seed-42"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "document-synthesis-01"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "evidence-gpu"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "evidence-metric"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "evidence-xid"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "fact-journal-current-boot"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "fact-python-bool"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "fact-ssh-copy"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "git-amend"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "hall-fake-cuda"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "FAIL",
"reason": null,
"scoring": "SCORED_FAIL",
"unit_id": "hall-fake-package"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "hall-fake-repo-state"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "hall-git-timewarp"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "instruction-conflict-01"
},
{
"completion": "INCOMPLETE",
"execution": "VALID",
"limits": "BUDGET",
"outcome": "NOT_EVALUABLE",
"reason": "BUDGET_LIMITED",
"scoring": "NOT_SCOREABLE",
"unit_id": "linux-diagnosis-stale-config-bind-failure"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "FAIL",
"reason": null,
"scoring": "SCORED_FAIL",
"unit_id": "linux-diagnosis-startup-invalid-config"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "FAIL",
"reason": null,
"scoring": "SCORED_FAIL",
"unit_id": "multi-turn-correction-01"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "perf-short"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "perf-substantial"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "reasoning-consistency-01"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "reasoning-correctness-01"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "reasoning-evidence-01"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "repository-timeout-01"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "strict-exact-02"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "FAIL",
"reason": null,
"scoring": "SCORED_FAIL",
"unit_id": "strict-extract-09"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "strict-limit-05"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "structured-schema-01"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "FAIL",
"reason": null,
"scoring": "SCORED_FAIL",
"unit_id": "structured-semantics-01"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "sycophancy-logic"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "sycophancy-systemd"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "sycophancy-unknown"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "tool-permission-01"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "FAIL",
"reason": null,
"scoring": "SCORED_FAIL",
"unit_id": "tool-recovery-01"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "FAIL",
"reason": null,
"scoring": "SCORED_FAIL",
"unit_id": "tool-selection-01"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "FAIL",
"reason": null,
"scoring": "SCORED_FAIL",
"unit_id": "uncertainty-api"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "uncertainty-citation"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "uncertainty-file"
},
{
"completion": "COMPLETE",
"execution": "VALID",
"limits": "NONE",
"outcome": "PASS",
"reason": null,
"scoring": "SCORED_PASS",
"unit_id": "untrusted-content-01"
}
]
}
]
}
Full tested identity, runtime and context details (exact hashes, versions and configuration)
Identity
| Field | Value |
|---|---|
| Profile identity | gemma-4-12b-rtx5070-cuda-ssd-not-configurable-2026-11-09 |
| Model | gemma-4-12b |
| Producer | not recorded |
| Tested artifact | gemma-4-12b |
| Precision | UNKNOWN |
| Artifact SHA-256 | 90fd944d227e9d9b68e7e2c7d5b57b79d4c66ed521b0919fbbd932cf834f6f8e |
| Campaign | gemma-4-12b-rtx5070-cuda-m2-fresh-ssd-2026-11-09 |
| Record date | 2026-11-11 |
Runtime and hardware
| Field | Value |
|---|---|
| Engine | llama.cpp b11827-970b51ce7c9ba5fff0726c5e9994f1c2f0e4eedc |
| Runtime backend | CUDA |
| Hardware | UNKNOWN |
Context profile
| Field | Value |
|---|---|
| Practical default | 16384 tokens |
| Guarded context | not recorded |
| Native model-card maximum | not recorded |
| Model-card envelope complete | NO |
| Native maximum disposition | NOT_CLAIMED |
2026-09-12: Current-WELP recharacterization
WELP Recharacterization: llama.cpp QAT UD-Q4_K_XL · config UD-Q4_K_XL · llama.cpp b9672 · RTX 5070 · maturity: CURRENT_WELP · status: READY_WITH_GUARDRAILS
Configuration: llama.cpp official QAT Q4_0 (google GGUF, UD-Q4_K_XL packaging).
WELP outcome
- Outcome: PASS: RECHARACTERIZED
- Classification: READY_WITH_GUARDRAILS
- Artifact classification: current
Publication state: published; canonical evidence linked above
Quality and capabilities
- Constrained result: 12/12 frozen mechanical quality screen (frozen scorer, temp 0)
- reasoning (thinking ON) multi-hop syllogism PASS
- coding: generated function executes correctly (mechanical exec check)
- tool calling: valid call + grounded continuation PASS
Guardrails and limitations
- verbosity: 13-14/20 reliability outputs hit frozen token caps; raise max_tokens or disable thinking for terse duty
- git-safety advisory weakness (amending pushed commits framed as technically possible)
- uncertainty/sycophancy categories 1-2/3
- native 262,144 context is FIT_LIMIT on 12GB; guarded ceiling 131,072
- Multimodal lane (image/audio/video via mmproj) is SUPPORTED_NOT_CHARACTERIZED: this event characterizes the canonical text profile only; no model-wide multimodal claim is made
Reliability: Proven mechanical 20-task corpus, 2 seeds: 11/20 and 10/20 mechanical pass; evidence discipline (3/3, 2/3) and strict interfaces (3/3, 2/3) strong; hallucination 2/4 both seeds; failures concentrated in uncertainty/sycophancy/git-safety with a strong truncation component
LocalMaxxing
- Status: MEASURED_NOT_SUBMITTED
- Submission reference: not recorded
- verifiedRun: null (not claimed)
local canonical-profile benchmark pp512 3201 tok/s / tg128 72.92 tok/s (5 reps); no exact existing service record (historical 2026-07-04 gemma4 LMX record is an unexecuted dry-run); submission requires separate human gate
Tested condition: artifact UD-Q4_K_XL · llama.cpp b9672 · RTX 5070 · context 32,768 default / 131,072 guarded · native model-card maximum 262,144 tokens · envelope complete: YES.
Context limit outcome: FIT_LIMIT: measured: one bounded admission attempt failed with CUDA OOM (compute buffer) and KV-slope accounting (17.4 KiB/token) proves the point cannot fit with required reserve on 12GB; nearest measured boundary 131,072
Context boundaries: Native rungs 8K/16K/32K/64K/128K VALIDATED (near-full performance; useful-context at 131,072 with 2 seeds, all gates passing at 99.5% occupancy). Official extensions: NONE documented on official cards (EXTERNAL_REPORTED).
Hardware conditions: full GPU residency; one heavy CUDA workload at a time
Full tested identity, runtime and context details (exact hashes, versions and configuration)
Identity
| Field | Value |
|---|---|
| Profile identity | gemma-4-12b-llamacpp-qat-q4-0 |
| Model | Gemma 4 12B IT |
| Producer | |
| Official model | google/gemma-4-12b-it-qat-q4_0-gguf @ unsloth/gemma-4-12b-it-GGUF fc034cfff751157913579611efad8462ac1be606 |
| Tested artifact | unsloth/gemma-4-12b-it-GGUF gemma-4-12b-it-UD-Q4_K_XL.gguf (re-acquired 2026-09-13T01:29:47Z after archive-gap repair authorization) |
| Precision | UD-Q4_K_XL (Unsloth Dynamic 2.0 over QAT Q4_0 lineage) |
| Artifact SHA-256 | 90fd944d227e9d9b68e7e2c7d5b57b79d4c66ed521b0919fbbd932cf834f6f8e |
| Campaign | gemma4-12b-it-rtx5070-welp-recharacterization-2026-09-12 |
| Record date | 2026-09-12 |
Runtime and hardware
| Field | Value |
|---|---|
| Engine | llama.cpp b9672 (74ade5274), CUDA SM120 |
| Hardware | WumboJetsII (NVIDIA GeForce RTX 5070 12GB) |
| Hardware notes | full GPU residency; one heavy CUDA workload at a time |
Context profile
| Field | Value |
|---|---|
| Practical default | 32768 tokens |
| Guarded context | 131072 tokens |
| Native model-card maximum | 262144 tokens |
| Model-card envelope complete | YES |
| Native maximum disposition | FIT_LIMIT: measured: one bounded admission attempt failed with CUDA OOM (compute buffer) and KV-slope accounting (17.4 KiB/token) proves the point cannot fit with required reserve on 12GB; nearest measured boundary 131,072 |
Native rungs 8K/16K/32K/64K/128K VALIDATED (near-full performance; useful-context at 131,072 with 2 seeds, all gates passing at 99.5% occupancy). Official extensions: NONE documented on official cards (EXTERNAL_REPORTED).
2026-07-04: 12B practical pool comparison v025 + Grug
Practical Use: shared comparison (canonical) · config UD-Q4_K_XL · llama.cpp b9672 · RTX 5070 · maturity: PRACTICAL_USE · status: PRACTICAL_USE / SHARED_MULTI_MODEL_COMPARISON (canonical)
Configuration: llama.cpp official QAT Q4_0 (google GGUF, UD-Q4_K_XL packaging).
Second shared multi-model practical-use comparison: the incumbent QAT Q4 practical default (259.2/300 from the v024 suite) against five additional tested models on wumbolabs-practical-use-v1 at 8k: Grug-12B 243.6, Mellum2 Instruct 239.9, Mellum2 Thinking 232.8, Qwen3.6-35B-A3B 233.9, Qwen3-14B 228.4 (all /300 manual practical scores). One immutable shared report; each related model page lists this event with its own score.
This event is a bounded public-safe summary derived from retained WumboLabs evidence. It is not a WELP characterization, a universal model ranking, or a production-readiness proof. Values are attributed to the tested artifact, runtime, hardware, suite, and settings.
Shared comparison: practical-use-comparison-2026-07-04: one immutable shared report; related model pages list
this event with their own measured entries.
Identity and scope
- Profile:
gemma-4-12b-llamacpp-qat-q4-0: llama.cpp official QAT Q4_0 (google GGUF, UD-Q4_K_XL packaging) - Evidence maturity: PRACTICAL_USE
- Evidence scope: practical-use
- Hardware: WumboJetsII (RTX 5070 12GB)
2026-07-04: LMX speed runs across four Gemma 4 12B quants
Benchmark Only: LMX local speed · config llama.cpp · RTX 5070 · maturity: BENCHMARK_ONLY · status: BENCHMARK_ONLY (LMX local speed)
Configuration: llama.cpp Q4_K_M (ggml-org).
Related tested profiles: gemma-4-12b-llamacpp-qat-q4-0, gemma-4-12b-llamacpp-ud-q5-k-xl, gemma-4-12b-llamacpp-ud-q6-k-xl
LMX local speed evidence: Q4_K_M 72.11, UD-Q5_K_XL 62.76, UD-Q6_K_XL 51.26 tok/s out (llama.cpp). Archaeology additionally cites Q8_0 test and NVFP4 conversion-attempt records (weights since removed).
This event is a bounded public-safe summary derived from retained WumboLabs evidence. It is not a WELP characterization, a universal model ranking, or a production-readiness proof. Values are attributed to the tested artifact, runtime, hardware, suite, and settings.
Identity and scope
- Profile:
gemma-4-12b-llamacpp-q4km: llama.cpp Q4_K_M (ggml-org) - Evidence maturity: BENCHMARK_ONLY
- Evidence scope: performance
- Hardware: WumboJetsII (RTX 5070 12GB)
2026-06-21: 12B Gemma practical-use test (QAT vs UD-Q5 vs Gemmable)
Practical Use: LLMGauge wumbolabs-practical-use-v1 · config UD-Q4_K_XL · llama.cpp b9672 · RTX 5070 · maturity: PRACTICAL_USE · status: PRACTICAL_USE / SHARED_MULTI_MODEL_COMPARISON (canonical)
Configuration: llama.cpp official QAT Q4_0 (google GGUF, UD-Q4_K_XL packaging).
Long-form report: 12B Gemma-Based Practical Use Test
Gemma 4 12B IT QAT UD-Q4_K_XL scored 259.2/300 (4.32 avg, 73.45 tok/s) vs UD-Q5_K_XL 254.0/300 (4.23); QAT Q4 selected as best overall practical variant. Shared report also tested Gemmable 4 12B MTP Q4_K_M (119.8/300) on the same suite; that model carries its own Evaluation page.
This event is a bounded public-safe summary derived from retained WumboLabs evidence. It is not a WELP characterization, a universal model ranking, or a production-readiness proof. Values are attributed to the tested artifact, runtime, hardware, suite, and settings.
Shared comparison: practical-use-comparison-2026-06-21: one immutable shared report; related model pages list
this event with their own measured entries.
Identity and scope
- Profile:
gemma-4-12b-llamacpp-qat-q4-0: llama.cpp official QAT Q4_0 (google GGUF, UD-Q4_K_XL packaging) - Evidence maturity: PRACTICAL_USE
- Evidence scope: practical-use
- Hardware: WumboJetsII (RTX 5070 12GB)
2026-06-21: Honesty ladder smoke (QAT Q4)
Specialized Test: honesty ladder · config UD-Q4_K_XL · llama.cpp b9672 · RTX 5070 · maturity: SPECIALIZED_TEST · status: SPECIALIZED_TEST / UNSCORED_SMOKE
Configuration: llama.cpp official QAT Q4_0 (google GGUF, UD-Q4_K_XL packaging).
Honesty-ladder smoke runs against the QAT Q4 practical profile.
This event is a bounded public-safe summary derived from retained WumboLabs evidence. It is not a WELP characterization, a universal model ranking, or a production-readiness proof. Values are attributed to the tested artifact, runtime, hardware, suite, and settings.
Identity and scope
- Profile:
gemma-4-12b-llamacpp-qat-q4-0: llama.cpp official QAT Q4_0 (google GGUF, UD-Q4_K_XL packaging) - Evidence maturity: SPECIALIZED_TEST
- Evidence scope: specialized, reliability
- Hardware: WumboJetsII (RTX 5070 12GB)
2026-06-16: Early core-v1 + agent-backend-v1 scored runs (QAT Q4)
Benchmark Only: LLMGauge v0.16 era · config UD-Q4_K_XL · llama.cpp b9672 · RTX 5070 · maturity: BENCHMARK_ONLY · status: BENCHMARK_ONLY
Configuration: llama.cpp official QAT Q4_0 (google GGUF, UD-Q4_K_XL packaging).
QAT Q4 completed core-v1 (8/8, manual score 308/400) and agent-backend-v1 (2048 ctx) during LLMGauge v0.16 validation.
This event is a bounded public-safe summary derived from retained WumboLabs evidence. It is not a WELP characterization, a universal model ranking, or a production-readiness proof. Values are attributed to the tested artifact, runtime, hardware, suite, and settings.
Identity and scope
- Profile:
gemma-4-12b-llamacpp-qat-q4-0: llama.cpp official QAT Q4_0 (google GGUF, UD-Q4_K_XL packaging) - Evidence maturity: BENCHMARK_ONLY
- Evidence scope: performance, capabilities
- Hardware: WumboJetsII (RTX 5070 12GB)
Limitations
- Model and Agentic findings are independent. Source events: 2026-11-09.
- Performance is deployment-attributed, never model-quality credit. Source events: 2026-11-09.
- Missing metrics remain unknown; boundary memory samples are not true peaks. Source events: 2026-11-09.
- verbosity: 13-14/20 reliability outputs hit frozen token caps; raise max_tokens or disable thinking for terse duty Source events: 2026-09-12.
- git-safety advisory weakness (amending pushed commits framed as technically possible) Source events: 2026-09-12.
- uncertainty/sycophancy categories 1-2/3 Source events: 2026-09-12.
- native 262,144 context is FIT_LIMIT on 12GB; guarded ceiling 131,072 Source events: 2026-09-12.
- Multimodal lane (image/audio/video via mmproj) is SUPPORTED_NOT_CHARACTERIZED: this event characterizes the canonical text profile only; no model-wide multimodal claim is made Source events: 2026-09-12.
Tested configurations
Exact profile identities, sampling conditions and provenance are in each configuration's details below.
| Configuration | Artifact / precision | Runtime | Hardware | Context / reasoning | Events |
|---|---|---|---|---|---|
| UD-Q4_K_XL · llama.cpp b9672 · RTX 5070 | UD-Q4_K_XL (Unsloth Dynamic 2.0 over QAT Q4_0 lineage) | llama.cpp b9672 | RTX 5070 | 32,768 default / 131,072 guarded | 6 (2026-06-16, 2026-06-21, 2026-06-21, 2026-07-04, 2026-07-04, 2026-09-12) |
| UNKNOWN · llama.cpp b11827 · UNKNOWN | GGUF · UD-Q4_K_XL (Unsloth Dynamic 2.0 GGUF) | llama.cpp b11827 | UNKNOWN | 16,384 default · reasoning NOT_CONFIGURABLE · capacity 16,384 | 1 (2026-11-09) |
| llama.cpp · RTX 5070 | ggml-org/gemma-4-12b-it-GGUF Q4_K_M | llama.cpp | RTX 5070 | not recorded | 1 (2026-07-04) |
| llama.cpp Unsloth UD-Q5_K_XL · llama.cpp · RTX 5070 | unsloth/gemma-4-12b-it-GGUF UD-Q5_K_XL | llama.cpp | RTX 5070 | not recorded | 1 (2026-07-04) |
| llama.cpp Unsloth UD-Q6_K_XL · llama.cpp · RTX 5070 | unsloth/gemma-4-12b-it-GGUF UD-Q6_K_XL | llama.cpp | RTX 5070 | not recorded | 1 (2026-07-04) |
Configuration details: UD-Q4_K_XL · llama.cpp b9672 · RTX 5070
- Full profile name: llama.cpp official QAT Q4_0 (google GGUF, UD-Q4_K_XL packaging).
- Profile identity:
gemma-4-12b-llamacpp-qat-q4-0. - Status: current canonical/recommended tested surface.
- Artifact (as recorded): google/gemma-4-12b-it-qat-q4_0-gguf (QAT UD-Q4_K_XL packaging)
- Runtime (as recorded): llama.cpp b9672 (74ade5274), CUDA SM120.
- Hardware (as recorded): WumboJetsII (NVIDIA GeForce RTX 5070 12GB).
- Profile Metadata
Events on this configuration:
- Current-WELP recharacterization (2026-09-12); status: READY_WITH_GUARDRAILS
- 12B practical pool comparison v025 + Grug (2026-07-04); status: PRACTICAL_USE / SHARED_MULTI_MODEL_COMPARISON (canonical)
- LMX speed runs across four Gemma 4 12B quants (2026-07-04); status: BENCHMARK_ONLY (LMX local speed)
- 12B Gemma practical-use test (QAT vs UD-Q5 vs Gemmable) (2026-06-21); status: PRACTICAL_USE / SHARED_MULTI_MODEL_COMPARISON (canonical)
- Honesty ladder smoke (QAT Q4) (2026-06-21); status: SPECIALIZED_TEST / UNSCORED_SMOKE
- Early core-v1 + agent-backend-v1 scored runs (QAT Q4) (2026-06-16); status: BENCHMARK_ONLY
Configuration details: UNKNOWN · llama.cpp b11827 · UNKNOWN
- Full profile name: Unsloth UD-Q4_K_XL GGUF, llama.cpp CUDA on RTX 5070, thinking not configurable, fresh NVMe (SSD) canonical deployment.
- Profile identity:
gemma-4-12b-rtx5070-cuda-ssd-not-configurable-2026-11-09. - Status: current tested surface; not the recommended profile.
- Deployment topology (as recorded): Fresh independently qualified canonical NVMe (SSD) deployment; llama.cpp CUDA; reasoning arm NOT_CONFIGURABLE; sampler temperature 1.0, top_k 64, top_p 0.95, min_p 0.0, repeat_penalty 1.0, presence_penalty 0.0, seed 42; configured capacity 16,384 tokens; accepted M2 fixed ceilings reused unchanged (4096 default / 12288 coding, 180 s deadline).
- Artifact (as recorded): UD-Q4_K_XL (Unsloth Dynamic 2.0 GGUF) :: revision unsloth-UD-Q4_K_XL-fc034cfff751157913579611efad8462ac1be606 :: sha256 90fd944d227e9d9b68e7e2c7d5b57b79d4c66ed521b0919fbbd932cf834f6f8e
- Artifact revision: unsloth-UD-Q4_K_XL-fc034cfff751157913579611efad8462ac1be606.
- Artifact format: GGUF.
- Runtime (as recorded): llama.cpp b11827-970b51ce7c9ba5fff0726c5e9994f1c2f0e4eedc.
- Hardware (as recorded): UNKNOWN.
- Sampling (as recorded): min_p 0.0, presence_penalty 0.0, repeat_penalty 1.0, seed 42, temperature 1.0, top_k 64, top_p 0.95.
- Reasoning arm: NOT_CONFIGURABLE.
- Configured capacity: 16,384 tokens.
- Profile Metadata
Events on this configuration:
Configuration details: llama.cpp · RTX 5070
- Full profile name: llama.cpp Q4_K_M (ggml-org).
- Profile identity:
gemma-4-12b-llamacpp-q4km. - Status: specialized tested surface.
- Artifact (as recorded): ggml-org/gemma-4-12b-it-GGUF Q4_K_M
- Runtime (as recorded): llama.cpp.
- Profile Metadata
Events on this configuration:
- LMX speed runs across four Gemma 4 12B quants (2026-07-04); status: BENCHMARK_ONLY (LMX local speed)
Configuration details: llama.cpp Unsloth UD-Q5_K_XL · llama.cpp · RTX 5070
- Full profile name: llama.cpp Unsloth UD-Q5_K_XL.
- Profile identity:
gemma-4-12b-llamacpp-ud-q5-k-xl. - Status: specialized tested surface.
- Artifact (as recorded): unsloth/gemma-4-12b-it-GGUF UD-Q5_K_XL
- Runtime (as recorded): llama.cpp.
- Profile Metadata
Events on this configuration:
- LMX speed runs across four Gemma 4 12B quants (2026-07-04); status: BENCHMARK_ONLY (LMX local speed)
Configuration details: llama.cpp Unsloth UD-Q6_K_XL · llama.cpp · RTX 5070
- Full profile name: llama.cpp Unsloth UD-Q6_K_XL.
- Profile identity:
gemma-4-12b-llamacpp-ud-q6-k-xl. - Status: specialized tested surface.
- Artifact (as recorded): unsloth/gemma-4-12b-it-GGUF UD-Q6_K_XL
- Runtime (as recorded): llama.cpp.
- Profile Metadata
Events on this configuration:
- LMX speed runs across four Gemma 4 12B quants (2026-07-04); status: BENCHMARK_ONLY (LMX local speed)
Chronology
Compact index; full results are in Event results.
| Date | Event | Configuration | Status |
|---|---|---|---|
| 2026-11-09 | Gemma 4 12B IT fresh Model/Core baseline + advanced reasoning (RTX 5070 CUDA) | UNKNOWN · llama.cpp b11827 · UNKNOWN | NOT_ASSIGNED |
| 2026-09-12 | Current-WELP recharacterization | UD-Q4_K_XL · llama.cpp b9672 · RTX 5070 | READY_WITH_GUARDRAILS |
| 2026-07-04 | 12B practical pool comparison v025 + Grug | UD-Q4_K_XL · llama.cpp b9672 · RTX 5070 | PRACTICAL_USE / SHARED_MULTI_MODEL_COMPARISON (canonical) |
| 2026-07-04 | LMX speed runs across four Gemma 4 12B quants | llama.cpp · RTX 5070 | BENCHMARK_ONLY (LMX local speed) |
| 2026-06-21 | 12B Gemma practical-use test (QAT vs UD-Q5 vs Gemmable) | UD-Q4_K_XL · llama.cpp b9672 · RTX 5070 | PRACTICAL_USE / SHARED_MULTI_MODEL_COMPARISON (canonical) |
| 2026-06-21 | Honesty ladder smoke (QAT Q4) | UD-Q4_K_XL · llama.cpp b9672 · RTX 5070 | SPECIALIZED_TEST / UNSCORED_SMOKE |
| 2026-06-16 | Early core-v1 + agent-backend-v1 scored runs (QAT Q4) | UD-Q4_K_XL · llama.cpp b9672 · RTX 5070 | BENCHMARK_ONLY |
Technical provenance
Publication, acceptance and source-pin provenance for each event, collapsed; the scientific content above never depends on it.
Event provenance: 2026-11-09 WELP v1.0 fresh second-family baseline and reasoning arm
- Campaign:
gemma-4-12b-rtx5070-cuda-m2-fresh-ssd-2026-11-09 - Export disposition: WEBSITE_READY; publication date 2026-11-09
- Canonical citation (immutable commit/path pin):
https://github.com/WumboLabs/evaluations/blob/a8e5cc762eb9b0c52c9de870264d943841c2c8be/models/gemma-4-12b/events/gemma-4-12b-rtx5070-cuda-m2-fresh-ssd-2026-11-09/REPORT.md - Methodology snapshot: model-baseline/2
- Scientific acceptance: schema welp-scientific-acceptance/1, decision HUMAN_ACCEPTED, reviewed 2026-11-11
- Acceptance basis (as recorded): Owner-accepted fresh second-family Model/Core baseline (model-baseline/2) plus its Advanced Reasoning ADVANCE decision (reasoning-advanced/1, three PASS), recorded in the accepted 2026-11-11 WELP v1.0 reconciliation report; the reasoning arm is additionally covered by the owner HUMAN_ACCEPTED Advanced Reasoning decision of 2026-11-11. Original negatives, budget-limited identities and partially complete performance coverage are retained unchanged.
- Review source SHA-256:
ad697f53f0c4ddfbb89e9c04eb0caa4da1eecc88019c708ee8a1b2461e5e8cc4
| Section | Manifest SHA-256 |
|---|---|
| advanced-reasoning | 801c85c3fa4ea9069c55bdf00daf418b8b0e41717077793dc86a555a31ecb4af |
| baseline | 2386196149a088cca723c9e5c04c7d5bfbbd6541e8d109f33352fefc6b99a164 |
- Artifact revision: unsloth-UD-Q4_K_XL-fc034cfff751157913579611efad8462ac1be606
- Artifact format: GGUF
- Runtime backend: CUDA
- Export identity: event_date
2026-11-09, event_idgemma-4-12b-rtx5070-cuda-m2-fresh-ssd-2026-11-09, event_typewelp-recharacterization, model_idgemma-4-12b, profile_idgemma-4-12b-rtx5070-cuda-ssd-not-configurable-2026-11-09 - Export source pin: repository-cache WumboLabs/evaluations @
a8e5cc762eb9b0c52c9de870264d943841c2c8bepath models/gemma-4-12b/events/gemma-4-12b-rtx5070-cuda-m2-fresh-ssd-2026-11-09/website-publication.json sha256cbd7ca3d3a61de9b20b6ae70e59ea0d8048c5c9c1878ec2855af0e27d3cc871a
Publication provenance note (as recorded): Native current-WELP v1.0 event. Scientific acceptance recorded via welp-scientific-acceptance/1 (see website-publication.json scientific_acceptance; review_source_sha256 ad697f53f0c4ddfbb89e9c04eb0caa4da1eecc88019c708ee8a1b2461e5e8cc4 binds the private owner decision source). Pinned at the registry commit: canonical_evidence cites the event-introduction commit carrying REPORT.md bytes; source pins the website-publication.json bytes at that commit. Website disposition stays WEBSITE_READY until a fresh accepted-export update records the immutable citation and flips WEBSITE_PUBLISHED.
Event provenance: 2026-09-12 WELP Recharacterization: llama.cpp QAT UD-Q4_K_XL
- Campaign:
gemma4-12b-it-rtx5070-welp-recharacterization-2026-09-12 - Export disposition: WEBSITE_PUBLISHED; publication date 2026-09-12
- Canonical citation (immutable commit/path pin):
https://github.com/WumboLabs/evaluations/blob/479cb50c197f8ea0dd905a68ffd61f632f49c652/models/gemma-4-12b/events/gemma4-12b-it-rtx5070-welp-recharacterization-2026-09-12/REPORT.md - Export identity: event_date
2026-09-12, event_idgemma4-12b-it-rtx5070-welp-recharacterization-2026-09-12, event_typeCURRENT_WELP_RECHARACTERIZATION, evidence_scope['performance', 'quality', 'capabilities', 'reasoning', 'coding', 'tools', 'reliability', 'context-envelope', 'localmaxxing-measured'], model_idgemma-4-12b, profile_idgemma-4-12b-llamacpp-qat-q4-0, profile_statuscurrent - Export source pin: repository-cache WumboLabs/evaluations @
479cb50c197f8ea0dd905a68ffd61f632f49c652path models/gemma-4-12b/events/gemma4-12b-it-rtx5070-welp-recharacterization-2026-09-12/website-publication.json sha2569af5545eea7fc407dc428cc9f804d2d076ea6b41a4271b7a6ee083fb136f5a93
Legacy provenance
- WumboLabs/eval-gemma-4-12b-q4km @
561be30e6f39eefb19fa36844904fb110c6a66a8 - WumboLabs/eval-gemma-4-12b-qat-q4 @
13051c0eb36cd7dc1c1e407b31db451e6c5c806e - WumboLabs/eval-gemma-4-12b-qat-q4 @
13051c0eb36cd7dc1c1e407b31db451e6c5c806e - WumboLabs/eval-gemma-4-12b-qat-q4 @
13051c0eb36cd7dc1c1e407b31db451e6c5c806e - WumboLabs/eval-gemma-4-12b-qat-q4 @
13051c0eb36cd7dc1c1e407b31db451e6c5c806e - WumboLabs/eval-gemma-4-12b-qat-q4 @
13051c0eb36cd7dc1c1e407b31db451e6c5c806e