OpenJev: Browser-Based LLM Evaluation Without Backend

What if I told you that running a full LLM evaluation suite could be as simple as opening a webpage? No servers to spin up, no API keys to manage, no cloud credits burning a hole in your budget. No waiting around for someone else's infrastructure to respond.

We've gotten used to thinking of serious model evaluation as inherently heavyweight. You need compute, you need coordination, you need at least a modest engineering effort just to keep the lights on. But what happens when you can run a 1.56GB model entirely client-side, in your browser, and still get meaningful comparisons?

The key insight here is that local models can take two very different approaches to constrained generation. They can either read the probabilities for your allowed options directly—without actually decoding them—or they can write the same distribution token by token, just like normal generation. Run both approaches on your own GPU, pick your model size, and measure the difference.

This isn't some corporate demo or academic paper behind a paywall. It's an independent research project called SemIf, and it's running entirely in your browser with no external dependencies. The question isn't whether this approach works—it's whether we've been overcomplicating evaluation from the start.

How OpenJev Works

OpenJev runs entirely in the browser through WebAssembly, which means the model loads directly into your system's memory and never touches a server. There's no backend infrastructure to spin up, no API keys to manage, and no network calls after the initial download. This is a genuinely different deployment model from most AI tools you're used to.

The tradeoff is upfront download size. A 1.56 GB MiniCPM5 2B model sits in your browser cache permanently after the first load, but subsequent sessions skip the download entirely. Here's what real-world performance looks like across three tested models:

| Model | Size | Authored | Perturbed | TypeSafe | |-------|------|----------|-----------|----------| | Qwen3 0.6B | 639 MB | 44.0% | 52.8% | 40.7% | | MiniCPM5 2B | 1.56 GB | 68.6% | 69.3% | 63.7% | | Qwen3.5 4B | 3.01 GB | 81.3% | 76.6% | 84.5% |

Larger models consistently outperform smaller ones, which tracks with what you'd expect from encoder-only architectures. The "Perturbed" benchmark seems to favor models that handle noise better, and Qwen3.5's 84.5% there is notable.

One thing that's genuinely confusing: the performance gap between "Authored" and "Perturbed" scores. A model can score 68.6% on authored data but jump to 69.3% on perturbed data, which suggests the perturbation isn't adding much difficulty. The TypeSafe benchmark, however, shows a wider spread — Qwen3 0.6B drops from 44.0% to 40.7%, while Qwen3.5 climbs from 81.3% to 84.5%.

import time

start = time.perf_counter()
first_response = jev.generate(prompt, max_new_tokens=1)
first_token_time = time.perf_counter() - start

print(f"First token: {first_token_time*1000:.1f}ms")

The GGUF format support means you can drop in any compatible model without conversion. It's not magic — you still need a CPU that doesn't suck and enough RAM to hold the model plus context. But it's refreshingly honest about what it is: a local inference engine that happens to run in your browser rather than on someone else's server.

Accuracy Benchmarks You Can Verify Yourself

The numbers here aren't theoretical. They're reproducible, which matters because most LLM benchmarks aren't. TypeSafe hits 40.7% agreement on the 102-row public subset, Authored scores 44.0%, and Perturbed reaches 52.8%. All three can be run locally with the included test data , no API calls, no hidden test sets.

Here's what the model lineup looks like:

The browser-only setup means no backend infrastructure, which also means no excuses about server-side optimizations skewing results. You download 1.56 GB on first load and that's it. I genuinely don't know how to feel about that trade-off , it's refreshingly transparent but also means you're carrying the full model weight in the browser.

The benchmark breakdown for Qwen3 0.6B shows the gap between human-authored responses and perturbed ones:

What's interesting is how the Perturbed score (52.8%) beats Authored (44.0%) on the smaller model. That suggests the perturbation process is catching something the original prompts miss , or the smaller model is just better at matching slightly-off references than perfectly-formed ones. Either way, it's the kind of result that makes you want to dig into the actual test data rather than nod along at a headline number.

The quote about "real choice vs fake choice" and the comparison to encoder-only models points to something deeper: the benchmark isn't just measuring accuracy, it's measuring how models handle ambiguity in inputs. That's harder to fake than a trivia question, and it's why these numbers feel more trustworthy than most.

Running Your First Evaluation

Loading a model starts with picking a file or pasting a URL—no signups, no backend configuration. The interface is pure browser, which means everything runs locally after that initial 1.56 GB download. You’ll see the model cached in local storage after the first load, so subsequent runs start faster.

Once your model is loaded, you select an evaluation dataset and hit run. Tokens stream in real time, and you can watch the output build character by character. The UI shows timing breakdowns: total time, input processing, and output generation. For example, running Qwen3 0.6B on the Authored benchmark gives you 44.0% accuracy, while MiniCPM5 2B hits 68.6% on the same dataset.

Results export as JSON or CSV with full token logs and timing data. You can inspect individual predictions, filter by subset (Authored, Perturbed, TypeSafe), and compare runs side by side.

import json

with open("run_results.json") as f:
    results = json.load(f)

for item in results["predictions"]:
    print(f"Input: {item['input']}")
    print(f"Output: {item['output']}")
    print(f"Expected: {item['expected']}")
    print(f"Match: {item['correct']}\n")

The streaming behavior is where this tool actually shines. Unlike batch-only evaluators, you see the model think—and stumble—in real time. That quote in the settings? “Give it a real choice / As opposed to a fake choice?” It’s not just flavor text. You’ll catch models hedging, refusing, or confidently generating the wrong token sequence, and those moments are harder to notice in static logs.

Practical Code Examples

The practical code examples reveal something I keep coming back to: owned columns are balanced accuracy. That's a mouthful, but it means the numbers you see aren't padded by selective reporting. TypeSafe shows equal-case agreement on the same 102-row public subset, and Jev's figures are straight from the published value. No hidden denominators.

I think this matters more than the community reaction suggests. Yes, there's confusion about unofficial emulations using smaller models—some of them slapdash knockoffs passing themselves off as Jev. But the core capability isn't about scale. It's about prompt specificity. Jev requires you to actually think about what you want before you ask for it. That's a constraint, not a flaw. Most AI tools are built to paper over ambiguity; Jev leans into it.

This is where I'm genuinely uncertain. The rapid innovation in synthetic-data-only training is real, and it does open doors for applications that can't rely on scraped web text. But whether that translates to broader adoption or stays a niche tool depends on how much friction users are willing to accept. I don't think Jev underestimates that friction—it knows exactly what it's asking for. The question is whether the rest of us are ready to meet it there.

Conclusion

The numbers don't lie, but they don't tell the whole story either. Qwen3 0.6B hits 44% balanced accuracy on that 102-row subset while MiniCPM5 2B climbs to 68.6% — a real gain, sure, but one that demands 1.56 GB of VRAM and a modern GPU. The model weights stay in your browser cache, inputs never leave the page, and there's something genuinely refreshing about an evaluation tool that doesn't ask for your data or your trust in return.

Still, I'm not sure what to make of OpenJev as a practical tool versus a proof of concept. It works — you can download, run, and verify results entirely client-side — but whether that matters outside of privacy-focused or air-gapped environments is genuinely unclear. The bigger question isn't whether browser-based LLM evaluation is possible; it's whether anyone will actually choose to run 1.5 GB models in Chrome when cloud APIs exist. Maybe that's not the point. Maybe the point is that the option now exists, and that's enough.