Local-Llm

Disclosure: Some links below are affiliate links. If you sign up through them, I may earn a commission at no extra cost to you. Vultr — high-memory cloud GPU instances from $6/mo DigitalOcean — $200 credit for new users, great for GPU droplets So Kimi K3 dropped two days ago and instantly became the most exciting open-weight model release this month — 7,300+ stars, 2.78 trillion parameters in a Mixture-of-Experts architecture, multimodal. Problem: nobody has a cluster sitting around to serve it. Enter Deltafin, a research project that does exactly what its README says: “runs a Mixture-of-Experts model far larger than the machine it sits on.” I spent the evening digging through its codebase, and honestly? The ambition matches the execution. ...

You’ve got a local LLM setup — Ollama, LM Studio, whatever. Now which model do you actually run? That’s the question nobody’s really answering well. HuggingFace shows you download counts. Ollama search tells you what fits in VRAM. But “fits” and “best” are two very different things. I’ve spent way too many afternoons downloading model after model, testing them one by one, only to wonder if there’s something better I missed. ...