Local LLM Baseline

This historical baseline predates the current WELP evaluation and publication methodology. It preserves the original test context; current model-testing method and claim rules live in Methodology.

Test Bench

Historical Baseline Philosophy

A model is only useful if it can fit the hardware, stay stable, follow instructions, avoid confident fabrication, and produce output that helps with real work.

For WumboLabs, that means testing against practical tasks:

• Linux troubleshooting
• Docker and networking diagnostics
• ZFS/recovery procedures
• configuration review
• local AI workflow support
• long-context behavior
• honesty under uncertainty

What gets tracked

Local model testing needs more than raw tokens per second.

Each useful benchmark record should include:

• model name
• quantization
• backend
• hardware
• context size
• prompt processing speed
• generation speed
• VRAM behavior
• fit/headroom notes
• failure modes
• practical usefulness
• whether the result is repeatable enough to trust

How to Read These Notes

Numbers are useful, but they are not enough.

A faster model that hallucinates commands, invents tools, or gives unsafe system advice is not a better model for this lab. The baseline is meant to capture both performance and judgment.

Later model-evaluation methodology is governed by WELP and the current evidence and publication rules in Methodology. This record remains a dated baseline, not the current testing entry point.