WUMBOLABS / EVALUATIONS CANONICAL MODEL EVALUATION

/evaluations/apodex-1.1-mini

Apodex 1.1 mini

Published evaluation evidence and testing history. Results remain attributed to their original configurations and methods.

latest evidence 2026-08-25

Overview

Apodex (Qwen3.5-35B-A3B MoE base) · 1 tested configuration · 1 evidence event · newest evidence 2026-08-25.

Contents: Findings · Event results · Tested configurations · Chronology · Technical provenance

Findings

Each finding is attributed to the event and tested configuration that measured it; exact values, failures and limitations are in the event results. A historical event's recorded status describes that event's own sections at its date and is not a verdict on newer sections.

  • Model classification: BOUNDED_EARLY_STOP / PHASE_3_DO_NOT_ADVANCE - attributed to the 2026-08-25 event, configuration IQ1_M · llama.cpp · RTX 5070.
  • Recommended configuration: llama.cpp IQ1_M (community conversion) (current): event result.
  • Latest evidence: 2026-08-25: event result
  • 2026-08-25 · IQ1_M · llama.cpp · RTX 5070: classification BOUNDED_EARLY_STOP / PHASE_3_DO_NOT_ADVANCE. Event result

Event results

Newest first. Each event is one independent testing/publication event with its full report linked once at the top of its section; scores, failures and limitations stay visible and raw section-provenance JSON stays collapsed.

2026-08-25: Initial evaluation (bounded early stop)

Initial Evaluation: Early Stop · config IQ1_M · llama.cpp · RTX 5070 · maturity: EARLY_STOP · status: BOUNDED_EARLY_STOP / PHASE_3_DO_NOT_ADVANCE

Configuration: llama.cpp IQ1_M (community conversion).

Full report (REPORT.md)

Hardware
FieldValue
MachineWumboJetsII
GPUNVIDIA GeForce RTX 5070 12GB (SM120)
VRAM12,227 MiB physical
CPUAMD Ryzen 7 9800X3D
OSFedora Linux 44
Driver / CUDANVIDIA 610.57.04
Runtimellama.cpp pinned at f280b26983ad0fdb705a0d9ebf0503e76f2899b0 (2026-08-24)
Headline Verdict

DO_NOT_ADVANCE at Phase 3 (early stop, frozen gate).

The model failed the applicable advancement gate. Per WELP protocol, later phases were not run. This is a bounded result, not a complete capability review.

The early-stop behavior is a feature of WELP transparency: the model failed the gate, so the evaluation stopped. This is not a protocol failure; it is the protocol working as designed.

Performance
MetricResult
Decode throughput169.36 tok/s (median 169.46, sigma 1.70)
Prefill (~5.6K tok)1,798.76 tok/s
TTFT (short prompt)~125 ms
VRAM idle9.7 GB
Phase 3 Practical Viability: DO_NOT_ADVANCE

Contract WELP Practical Viability 0.1.2-draft, scorer score_pv.py (self-test 35/35), 30 tasks x seeds {42,43,44}, thinking OFF baseline.

GateThresholds42s43s44Verdict
G1 aggregate>=0.750.6670.7000.733FAIL (all)
G2 false-premise>=0.600.4000.2000.600FAIL (42,43)
G3 factual-uncertainty>=0.500.5000.7500.500PASS (boundary)
G4 structured-output>=0.700.7140.7140.714PASS

Decision identical across seeds => stable FAIL. Bounded failure review: ~19 genuine failures (fabricated package description, explained nonexistent git flag, asserted that 25 is prime, invented phone number, echoed un-reversed word); ~8 scorer lexicon artifacts. Verdict robust: correcting every artifact still leaves seed-42 aggregate <0.75.

What Was NOT Tested
  • Native Tools module
  • Coding module
  • Reasoning module (dedicated)
  • Context / Variance / Optimization / Stability
  • Soak
  • OMP Stage 2 (agent)

Do not fabricate missing results. This evaluation establishes that the strongest representation satisfying the frozen WumboJetsII full-GPU baseline (IQ1_M, 1.75-bpw) does not clear the frozen viability gate. It does NOT establish that BF16/FP8/GPTQ or Agent Team deployments would fail.

Role Classification

Supported roles: none beyond "runs coherently on 12GB at extreme quantization."

NOT_REACHED: Phases 4–10, OMP Stage 2, all capability modules, context/variance/optimization/soak/classification engine.

Important Limitations
  • Extreme quantization forced by hardware: single 12GB consumer GPU required IQ1_M (1.75-bpw). Producer positioning rests on BF16-class deployments plus an agent harness that was not authorized to run.
  • Phase-3 failure at IQ1_M does NOT establish that BF16/FP8/GPTQ or Agent Team deployments would fail. It establishes that the strongest representation satisfying the frozen full-GPU baseline does not clear the frozen viability gate.
  • Scorer lexicon defect understates false-premise performance modestly; verdict unaffected.
  • No reliability, coding, tools, context, or soak results exist.
  • Canonical evaluation repo: https://github.com/WumboLabs/evaluations/tree/4f93731a9f042b9c1d86718a98a4e0e13d0fbd21/models/apodex-1.1-mini/profiles/apodex-1.1-mini-llamacpp-iq1m
  • WELP protocol: https://github.com/WumboLabs/welp
  • Labs catalog: https://github.com/WumboLabs/evaluations/tree/4f93731a9f042b9c1d86718a98a4e0e13d0fbd21/models
  • LocalMaxxing: Speed result SUBMITTED/APPROVED (ID cmt9ijytg00xali017f46xk25, 182.31 tok/s p512/n128). Benchmark suites NOT_SUBMITTED due to early WELP stop.
Reproduction

Direct link to canonical reproduction material: Central evidence

This Lab Record is a summary; the canonical repo is the source of truth.

Identity
FieldValue
ModelApodex 1.1 mini
ProducerApodex; base: Qwen/Qwen3.5-35B-A3B; ~36B total / A3B active MoE
Evaluated artifactCommunity conversion abenzerps/Apodex-1.1-mini-GGUF @ 59afa57852525f79a9634d5f80dd639cceee572c (not official)
FileApodex-1.1-mini-IQ1_M.gguf
SHA-2561e84d8adf7837e96fb18712882a8a114becc7e53554372e2f612c5e0c6276cd4
Evaluation date2026-08-25
ProtocolWELP end-to-end (frozen snapshot welp-next-snapshot-2026-08-25-end-to-end)
Highest phase reachedPhase 3 (gate decision)
Campaign classificationBOUNDED_EARLY_STOP / PHASE_3_DO_NOT_ADVANCE

Tested configurations

Exact profile identities, sampling conditions and provenance are in each configuration's details below.

ConfigurationArtifact / precisionRuntimeHardwareContext / reasoningEvents
IQ1_M · llama.cpp · RTX 5070IQ1_Mllama.cppRTX 5070not recorded1 (2026-08-25)
Configuration details: IQ1_M · llama.cpp · RTX 5070
  • Full profile name: llama.cpp IQ1_M (community conversion).
  • Profile identity: apodex-1.1-mini-llamacpp-iq1m.
  • Status: current canonical/recommended tested surface.
  • Artifact (as recorded): Apodex-1.1-mini-IQ1_M.gguf (community conversion abenzerps/Apodex-1.1-mini-GGUF @ 59afa57852525f79a9634d5f80dd639cceee572c)
  • Runtime (as recorded): llama.cpp.
  • Profile Metadata

Events on this configuration:

Chronology

Compact index; full results are in Event results.

DateEventConfigurationStatus
2026-08-25Initial evaluation (bounded early stop)IQ1_M · llama.cpp · RTX 5070BOUNDED_EARLY_STOP / PHASE_3_DO_NOT_ADVANCE

Technical provenance

Publication, acceptance and source-pin provenance for each event, collapsed; the scientific content above never depends on it.

Model evidence tree (GitHub)

Legacy provenance