Topic: client-side ai

OpenJev: Browser-Based LLM Evaluation Without Backend

OpenJev: Browser-Based LLM Evaluation Without Backend
What if I told you that running a full LLM evaluation suite could be as simple as opening a webpage? No servers to spin up, no API keys to manage, no cloud credits burning a hole in your budget. No waiting around for someone else's infrastructure to respond. We've gotten used to thinking of serious model evaluation as inherently heavyweight. You need compute, you need coordination, you need at least a modest engineering effort just to keep the lights on. But what happens when you can run a 1.56GB model entirely client-side, in your browser, and still get meaningful comparisons? The key insight here is that local models can take two very different approaches to constrained generation. They can either read the probabilities for your allowed options directly—without actually decoding them—or they can write the same distribution token by token, just like normal generation. Run both approaches on your own GPU, pick your model size, and measure the difference. This isn'...