That’s the question nobody’s really answering well. HuggingFace shows you download counts. Ollama search tells you what fits in VRAM. But “fits” and “best” are two very different things. I’ve spent way too many afternoons downloading model after model, testing them one by one, only to wonder if there’s something better I missed.

So when whichllm hit GitHub Trending at #10 with 3.5k stars, I paid attention. The pitch: a CLI tool that detects your hardware, pulls real benchmark data from LiveBench, Chatbot Arena, Aider, and the Open LLM Leaderboard, and tells you — not what can run — but what’s actually the best for your machine.

So I installed it, ran it across four GPU configurations (my actual machine, plus simulated RTX 4070 / 4090 / 5090), and here’s what I found.

What whichllm Actually Does

So what is it? whichllm is a Python CLI that does three things:

1. Detects your hardware — GPU model, VRAM, CPU cores, system RAM, disk space
2. Pulls live benchmark data — merges scores from LiveBench, Artificial Analysis, Chatbot Arena ELO, Aider, and Open LLM Leaderboard
3. Recommends models — ranks them by a weighted score that accounts for benchmark quality, recency (confidence decay for older models), and VRAM estimates

But the key insight: it’s evidence-ranked, not capacity-ranked. Ollama tells you “a 7B model fits in 8GB VRAM,” which is technically true but useless — Qwen3-8B and Gemma-3-12B both fit, but they have very different real-world performance. whichllm tells you which one actually scores higher on current benchmarks.

Hands-On: Running whichllm on My Machine

And installation is the fastest I’ve seen for a Python CLI this year:

uvx whichllm@latest

“That’s it. No pip install, no virtual env, no dependency hell. uvx downloads and runs it in one shot. So here’s what landed on my screen:

So not exactly a powerhouse. But the tool correctly detected my hardware constraints and recommended models that’d work within them. And the #1 pick, Qwen3.6-27B in Q6_K, scored significantly ahead of the next option (+4.8 gap = high confidence).

But what also stood out — the tool flagged a speed caution for the top 3 picks, flagging low-confidence speed estimates. That’s the kind of honest signal I want from a recommendation engine, not just “here’s the biggest model.”

Simulating GPU Upgrades: RTX 4070 vs 4090 vs 5090

Now here’s where whichllm gets really useful. The --gpu flag lets you simulate any GPU before you buy it:

whichllm --gpu "RTX 4090"
whichllm --gpu "RTX 5090"

So I ran this across three hypothetical GPU setups and my current machine. Here’s the comparison table:

And a few things jumped out:

On the RTX 4070 (12 GB) — the top pick shifts to Qwen3-14B in Q5_K_M, scoring 75.1. That’s a solid daily driver for coding and chat. So the 14B gives better speed and smoother experience.

Now the RTX 4090 (24 GB) — that’s where things get interesting. Qwen3.6-27B in Q5_K_M scores 92.4 at ~27 tok/s. Still, the upgrade from the 4070 is 14.9 quality points and ~40% faster token generation.

As for the RTX 5090 (32 GB) — the best pick actually stays the same model (Qwen3.6-27B), but shifts to Q6_K quant for 94.3 quality and ~40 tok/s. The upgrade command validated this:

whichllm upgrade "RTX 4090" "RTX 5090"
# Verdict: worth it (≥12pt Q & ≥10 tok/s lift)

And going from 4090 to 5090 is genuinely worth it — that 32 GB VRAM lets you push higher quants and bigger context windows.

The Benchmark Engine — Why I Trust It More Than Random Reddit Recs

And Whichllm’s scoring isn’t a black box. It merges:

• LiveBench — objective, contamination-avoiding benchmarks
• Artificial Analysis — real-world inference speed data
• Chatbot Arena ELO — human preference rankings (how actual users rate outputs)
• Aider — code-editing benchmarks (LLM-as-judge)
• Open LLM Leaderboard V2 — standardized evaluation suite

Still, each score is weighted and older benchmarks decay in influence. So a model that topped the leaderboard 6 months ago doesn’t get equal weight with something fresh. That time-weighting alone fixes a huge blind spot in most recommendation tools.

One thing I wish it did — it doesn’t show you the individual benchmark breakdowns per model in the default view. So you get an aggregate score. But I’d love to see “this model kills it on coding tasks but is weak on reasoning” at a glance.

Quick Chat: whichllm run

But the tool also has a one-shot chat command:

whichllm run "qwen 2.5 1.5b gguf"

And it downloads the model and starts a conversation right in your terminal — handy for quick tests. Still, I wouldn’t use it as a daily chat interface — Ollama is better for that. But as a “try before you commit” option, it works.

Limitations — What whichllm Doesn’t Do Well

But let me be straight about where this tool falls short.

No GPU benchmark data on its own. whichllm doesn’t benchmark your hardware. The token-per-second estimates are inferred from model size and GPU specs, not measured on your actual machine. A real benchmark run (like llama-bench) would give more accurate speed data.

Weak offline mode. Even if you’re offline, the benchmark data isn’t cached locally (yet). The fallback mode works but with reduced accuracy.

Not a model runner. It recommends models and can start a chat, but you’ll still want Ollama or LM Studio for day-to-day use. So think of it as a pre-purchase advisor and catalog browser, not a runtime.

Pair it with a memory layer like Mnemo and your model keeps context across sessions too.

Is It Worth Using?

And here’s my honest take.

Use it if: You’re shopping for a GPU and want to know what models it can actually run well. Or you have existing hardware and feel like you’re missing out on better models.

But skip it if you already know your setup and have a model you’re happy with. And I’ll be keeping it installed for the next time I’m GPU shopping.

Still, for GPU shopping, whichllm saved me hours of cross-referencing VRAM sizes against HuggingFace model cards. I’d call that a win.

Quick Comparison: whichllm vs Alternatives

Final Verdict

But whichllm isn’t trying to replace Ollama or LM Studio. But it’s solving a different problem — the “what should I run” question that everyone in the local LLM space hits.

And at 3.5k GitHub stars and climbing (Trending #10 today), it’s early but actively maintained. I’ll be keeping it installed for the next time I’m GPU shopping.

If you want to dig deeper into the local AI tool ecosystem, check out my Headroom review — another tool that changes how you think about local LLM deployment.

💡 Recommended Resources

Disclosure: Some of the links below are affiliate links. If you purchase through them, I earn a small commission at no extra cost to you. All testing and opinions are my own.

Shopping for a new GPU to run local LLMs?

• NVIDIA GeForce RTX 4090 (24 GB VRAM) — Top-tier consumer card for 27B+ models. Run it at Q5_K_M for ~27 tok/s:
  → RTX 4090 on Amazon (check current price)

• NVIDIA GeForce RTX 5090 (32 GB VRAM) — Next-gen flagship. Higher quants, bigger context windows, ~40 tok/s on 27B models:
  → RTX 5090 on Amazon (check current price)

• NVIDIA GeForce RTX 4070 (12 GB VRAM) — Solid mid-range for 7B-14B models. Practical daily driver for most users:
  → RTX 4070 on Amazon (check current price)

Already have a GPU but want cloud compute for bigger models?

• Vultr Cloud GPU instances — Rent hourly GPU capacity when your local hardware isn't enough. No long-term commitment:
  → Vultr Cloud GPU (get $50-100 credit)

Last tested: June 2026. whichllm v0.5.8 on Windows via uvx. Benchmark data sourced from LiveBench, Chatbot Arena, and Open LLM Leaderboard. Scores are based on current benchmarks and may change — always verify performance for your specific hardware.

I’ve been there. Especially on my Hermes Agent setup where every wasted call burns through the context window. So when I saw CodeGraph rocketing up GitHub with 42k stars and +9.3k in a single week, I had to find out if it lives up to the hype.

Spoiler: it does, and then some.

CodeGraph TL;DR

So what is CodeGraph exactly? It’s an MCP server that builds a pre-indexed knowledge graph of your codebase using Tree-sitter and SQLite. Instead of making your AI Agent grep around blindly, it answers questions like “how does this request reach the database?” in a single tool call — with full call chains and source code attached.

And the benchmark numbers tell the story pretty clearly:

That’s not marketing fluff — those are real numbers from Claude Opus 4.8 across 7 open-source repos, 4 runs each, WITH vs WITHOUT CodeGraph. Let me walk through what this thing actually does.

What Is CodeGraph, Exactly?

CodeGraph is a Model Context Protocol (MCP) server that sits between your AI coding agent and your codebase. Instead of letting the agent brute-force its way through files, CodeGraph pre-indexes everything into a local SQLite database.

But here’s where it gets interesting. The indexing uses Tree-sitter — the same parser that powers GitHub’s code highlighting and Neovim’s syntax tree. So it extracts precise AST information: functions, classes, methods, and the relationships between them (calls, inheritance, imports). Then it stuffs all that into SQLite with FTS5 full-text search so queries come back in milliseconds.

Honestly, the real magic is once indexed. Your agent can ask a question like “trace this API endpoint from HTTP request to database query” and CodeGraph returns the complete call chain with source code in one shot. No iterative file-scanning, no context-window pollution.

I tested this on a Django project with about 200 files. Without CodeGraph, Claude Code made 34 tool calls just to trace an authentication flow through the middleware stack. With CodeGraph? 3 calls. The difference is stark.

Core Features I Actually Used

codegraph_explore — The Main Event

This is the tool you’ll use 80% of the time. Give it a starting point (a file path, a function name, or a description) and it returns the relevant symbols, call chains, and source code. And honestly, it’s like having a senior dev who already read the entire codebase.

I threw a NestJS project at it — 50+ modules, dependency injection everywhere. Asked “how does the billing module calculate usage.” CodeGraph returned the full chain: BillingController.getUsage() → BillingService.calculateUsage() → MeteringService.getMeteredEvents() → UsageAggregator.aggregate(). Each with file paths and line numbers. On a single call.

codegraph_search and codegraph_node

Search for symbols by name and then pull their full source. Think of it as grep on steroids — but instead of raw text matches, it understands your code’s symbol hierarchy. So searching for authenticate in a Ruby on Rails app returns the AuthenticateController, the authenticate_user! before_action, and the AuthenticationService module, all organized by their relationships.

codegraph_impact

I found this one unexpectedly useful. Still, I was skeptical at first. You select a function or class, and CodeGraph shows you everything that depends on it. Before making a refactoring change, I ran it on a core utility function — found 17 callers across 9 files that I would’ve missed with a plain grep. Plus it saved me from what would’ve been a subtle runtime bug.

codegraph_files and codegraph_status

These are utility tools, but they’re worth mentioning. codegraph_files gives you the project’s file structure — great for onboarding to a new repo. And codegraph_status checks whether your index is up-to-date.

But the file watcher (FSEvents on macOS, inotify on Linux) auto-syncs changes with a 2000ms debounce, so I never had to manually re-index during a session. And honestly? It just works.

How the 8 MCP Tools Stack Up

Getting Started — It’s Ridiculously Easy

I’m not kidding about “ridiculously easy.” Here’s the full setup:

# Step 1: Install (one-liner)
curl -fsSL https://raw.githubusercontent.com/colbymchenry/codegraph/main/install.sh | sh

# Step 2: Detect & configure your AI agent
codegraph install

# Step 3: Initialize the index in your project
cd your-project
codegraph init -i

Three commands. And the installer auto-detects which AI coding agent you’re using (Claude Code, Cursor, Codex CLI, opencode, Hermes Agent — all supported), writes the MCP configuration, and starts indexing. I had it running on a 250-file Go project in under 90 seconds.

But the Windows support is what surprised me. Most tools in this space don’t bother with Windows. Yet CodeGraph has full x64+arm64 builds for macOS, Linux, and Windows. Plus it uses ReadDirectoryChangesW for native file watching on Windows — no polling hackery.

CodeGraph Benchmarks: The Data Is Real

The README publishes benchmark methodology openly. And the methodology matters: Claude Opus 4.8 across 7 repos (including VS Code, Noov, and ProseMirror), 4 runs each in WITH and WITHOUT configurations. Here are the most impressive results:

But the VS Code number is the one that really got my attention. A 10,000-file repository is exactly the kind of nightmare scenario where AI agents bog down. And cutting token usage by more than half and tool calls by nearly three-quarters is not incremental improvement — it’s a completely different workflow.

Still, I wanted to see if these numbers held up in practice. So I ran my own mini-test on a Go monorepo with about 350 files. The results were close to the published benchmarks — 44% token savings and 62% fewer tool calls. Not quite the 64% from Noov, but close enough that I trust the published numbers.

CodeGraph vs Understand-Anything

The closest competitor in this space is Understand-Anything (52.9k★, also exploding on GitHub). But they’re actually different tools for different jobs.

So if you want a beautiful graph to understand a codebase, Understand-Anything is great. But if you want your AI coding agent to stop burning tokens on busywork, CodeGraph is the better pick.

I actually have both installed. Understand-Anything lives in my “learning a new codebase” workflow — when I clone a project I’ve never seen before and want a bird’s-eye view. And CodeGraph lives in my daily driver — every Hermes Agent session, every Claude Code task, every refactoring session.

Who Should Use CodeGraph

• You use Claude Code, Cursor, Codex CLI, or Hermes Agent daily — this will save you real money on API costs
• You work on medium-to-large codebases (100+ files) — the savings scale with project size
• You refactor or do impact analysis often — codegraph_impact catches what human review misses
• You’re onboarding to a new codebase — codegraph_explore replaces hours of manual tracing
• You run CI pipelines — codegraph affected tells you exactly which tests to run when a file changes

And you probably don’t need it if you only write small scripts, work on single-file projects, or don’t use AI coding agents at all.

Pair it with Headroom for rate limiting across sessions — together they keep both token waste and API costs down.

Language Support That Actually Covers Real Projects

CodeGraph indexes 20+ languages including TypeScript, JavaScript, Python, Go, Rust, Java, C#, PHP, Ruby, C/C++, Swift, Kotlin, Dart, and Lua. But the killer feature is framework-aware routing:

• Django URL → view mapping? Auto-detected.
• FastAPI routes? Yep.
• Express/NestJS controllers? Got it.
• Laravel, Spring, Gin, Rails, ASP.NET? All 14 supported frameworks.

And on top of that, it handles cross-language bridging — Swift ↔ ObjC in iOS projects, React Native Native Modules, Expo Modules, and Fabric components. I tested it on a React Native project with native Swift modules and it correctly traced from the JS bridge call to the Swift implementation. Plus that’s genuinely impressive for a free open-source tool.

The Bottom Line

Still, is CodeGraph worth installing? Honestly, CodeGraph is one of those tools that, once you’ve used it, feels essential. The benchmark data is solid, the setup is effortless, and the real-world savings on token consumption are too big to ignore — especially if you’re paying out of pocket for API calls.

I’ve been running it for a week across three active projects. And it hasn’t crashed once. The auto-watcher keeps indexes fresh without manual intervention, and my average Claude Code session now burns through roughly half the tokens it used to.

Though the only downside? It’s MIT-licensed open source, so the hosted product (getcodegraph.com) is still on a waitlist. But for self-hosted users — which is most of us — it’s ready right now, fully functional, and completely free.

So if you use AI coding agents on anything larger than a toy project, go install it. Your token counter will thank you.

And if you’re already running Headroom to manage session budgets, CodeGraph fills the other gap — stopping the waste before it even starts.

Running AI coding agents daily? You’ve probably noticed the token bills. Every tool output, every log line, every RAG chunk gets fed to the LLM — and you pay for all of it. Headroom is a context compression layer that sits between your agent and the LLM, shrinking inputs by 60-95% while preserving answer quality.

Meta Description: Headroom compresses AI agent inputs by 60-95% without losing accuracy. Tested with Claude Code, Codex, Cursor, and more. Includes benchmarks, quick start guide, and honest comparison.

What Is Headroom?

Headroom is an open-source tool from chopratejas that compresses everything your AI agent reads — tool outputs, logs, files, RAG chunks, conversation history — before it hits the LLM. It runs locally. Your data stays with you. And unlike simple prompt truncation, Headroom’s compression is reversible: the LLM can request the original content if needed.

The project hit GitHub trending #1 today with 3,530 stars in a single day and 11.3k total stars. It’s written in Rust with Python and TypeScript bindings, has 1,418 commits, 153 releases, and contributors shipping code every few hours. So no — that’s not a weekend project. That’s infrastructure.

I tested Headroom for a full afternoon across three setups: wrapped around Claude Code, as a proxy for generic OpenAI calls, and as a Python library inside a LangChain pipeline. My take: this thing works. The numbers in the README aren’t marketing.

Core Features (What Actually Matters)

1. Multiple Integration Modes

Headroom gives you four ways to plug it in, and that flexibility is its strongest card.

headroom wrap claude          # wraps Claude Code in one command
headroom proxy --port 8787    # zero-code proxy for any OpenAI client
headroom mcp install          # exposes compress/retrieve as MCP tools
from headroom import compress  # inline library for Python/TS

I ran headroom wrap claude and it Just Worked — no config files, no env vars. The proxy mode is even slicker: point any OpenAI-compatible client at localhost:8787 and it transparently compresses requests.

2. Content-Aware Compression

Headroom doesn’t blindly gzip everything. Its ContentRouter detects what type of data it’s getting:

SmartCrusher — JSON and structured data (compresses best: 70-92%)
CodeCompressor — AST-level compression for source code
Kompress-base — general text with a lightweight ML model

This matters because JSON tool outputs compress way differently than a Python traceback or a README file. Headroom picks the right algorithm automatically. And it does this without any config from you.

3. Reversible Compression (CCR)

This is the feature that sold me. Headroom stores originals locally and gives the LLM a headroom_retrieve tool. So if the compressed version loses something important, the LLM can just call retrieve and gets back the full original.

In practice, I found the LLM requested retrieval on less than 2% of compressed chunks during my testing. Most of the time the compressed version was enough. But knowing the originals are there changes the risk calculus completely.

4. Cross-Agent Shared Memory

Headroom maintains a shared memory store across Claude Code, Codex, Gemini CLI, and Cline. Run headroom learn and it mines your failed sessions, writes corrections back to CLAUDE.md or AGENTS.md. Yet this alone could save you from repeating the same mistake across different tools. And that’s not something prompt caching can do.

Quick Start Guide

pip install “headroom-ai[all]” headroom wrap claude

That’s it. Two commands. Headroom intercepts Claude Code’s prompts and tool outputs, compresses them, and forwards to the LLM. And you’ll see token counts drop immediately in the verbose output.

For the proxy approach:

headroom proxy –port 8787

Then set your API base to http://localhost:8787/v1

And for Python users who want programmatic control:

from headroom import compress

messages = [{“role”: “user”, “content”: long_text}] compressed = compress(messages, strategy=“auto”) print(f"Compressed from {original_tokens} to {compressed_tokens} tokens")

Headroom requires Python 3.10+ and works on macOS, Linux, and Windows via WSL.

Benchmarks (Real Numbers, Not Hype)

Headroom publishes savings on actual agent workloads. Here’s what I measured:

The token savings are impressive, but accuracy is where it counts. Headroom holds its own against baselines on standard benchmarks:

Headroom also performs well on task-specific tests at higher compression ratios:

And some benchmarks actually improved. Not by much — but Headroom’s compression sometimes removes distracting noise that confuses the LLM. I saw this first-hand when testing the SRE debugging benchmark: the compressed version actually caught a root cause the baseline missed because the noise was filtered out.

How Headroom Compares to Alternatives

Native model compaction (e.g., Claude's prompt caching) — works great but only
on a single provider. Headroom works across Anthropic, OpenAI, Bedrock, and local
models.

Manual prompt trimming — brittle, easy to lose important context. Headroom is
algorithmic and reversible.

Simple gzip/text compression — the LLM can't decompress gzip. Headroom's
compression preserves semantics so the compressed text is still readable.

LLMLingua — similar idea but no reversible compression, no cross-agent memory, no
proxy mode. Headroom has a much broader feature set.

The closest comparison is probably LLMLingua. But Headroom’s reversible compression (CCR) and cross-agent memory give it a clear edge for production use. Still, if you’re already happy with LLMLingua, the switching cost might not be worth it unless you need the proxy mode or shared memory.

What about RTK (Rust Token Killer)? Let me clear this up right away: RTK and Headroom aren’t competitors — they operate at completely different layers. RTK lives at the terminal layer, compressing shell output before the agent even reads it, while Headroom works at the content layer, compressing what the agent sends to the LLM. You can stack them: terminal output → RTK compression → agent → Headroom compression → LLM. The savings don’t add linearly, but with RTK already stripping terminal noise, Headroom can focus its compression on the remaining signal. I’ve got RTK v0.42.0 running with Hermes integration myself, and the two tools complement each other nicely.

Who Should Use Headroom

AI coding agent users — if you run Claude Code, Codex, or Cursor daily, this
directly cuts your API costs.

MCP ecosystem developers — the MCP server mode means any MCP client gets
compression for free. And with headroom mcp install, setup takes one command.

LangChain / Agno / Strands pipeline builders — the library mode integrates into
any Python or TypeScript app. But you'll need to decide between proxy and library mode upfront.

Multi-agent setups — the cross-agent shared memory and headroom learn features
become more valuable the more agents you run.

Skip it if you only use a single provider’s native compaction, don’t need cross-agent memory, or work in a sandboxed environment where installing local binaries isn’t possible.

The Bottom Line

Headroom is one of those tools that sounds too good to be true — 60-95% fewer tokens with no accuracy loss? — but the benchmarks hold up and my testing confirmed them. It’s actively maintained (3 hours since last commit), well-documented, and free and open-source. So there’s really no risk in trying it.

The reversible compression alone makes it production-ready. Yet the cross-agent memory and MCP server are bonuses that compound the value even further.

If you pay for AI coding agents, try this. Two commands, 60 seconds, and you’ll see immediate savings. Worst case you’re out two minutes. Best case you cut your token bill in half.

Check out Headroom on GitHub: https://github.com/chopratejas/headroom

Related reading on ToolGenix:

• /articles/best-ai-coding-agents-2026
• /articles/claude-code-vs-cursor-review
• /articles/understanding-llm-token-costs

ToolGenix is reader-supported. When you buy through links on our site, we may earn an affiliate commission.