<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>CEL Rules on ToolGenix — Open-Source AI &amp; Developer Tools: Honest Hands-On Reviews</title>
    <link>https://toolgenix.nxtniche.com/tags/cel-rules/</link>
    <description>Recent content in CEL Rules on ToolGenix — Open-Source AI &amp; Developer Tools: Honest Hands-On Reviews</description>
    <generator>Hugo</generator>
    <language>en-us</language>
    <lastBuildDate>Thu, 30 Jul 2026 00:00:00 +0000</lastBuildDate>
    <atom:link href="https://toolgenix.nxtniche.com/tags/cel-rules/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>Numbat Review: Agent Activity Monitor From Perplexity AI</title>
      <link>https://toolgenix.nxtniche.com/posts/numbat-perplexity-agent-activity-monitor-review/</link>
      <pubDate>Thu, 30 Jul 2026 00:00:00 +0000</pubDate>
      <guid>https://toolgenix.nxtniche.com/posts/numbat-perplexity-agent-activity-monitor-review/</guid>
      <description>Hands-on with numbat: Perplexity AI&amp;#39;s open-source agent monitor. I tested it for 90 minutes, caught 3 rogue file reads, and verified CEL blocking rules.</description>
      <content:encoded><![CDATA[<p>Ever had an AI coding agent go rogue — read a file it shouldn&rsquo;t, fire off an API call you didn&rsquo;t approve, or quietly delete something important?</p>
<p>Yeah, me too.</p>
<p>That moment when you realize your Claude Code session silently touched <code>/etc/passwd</code> or your Codex agent issued a <code>curl</code> to an IP you&rsquo;ve never seen before — you can&rsquo;t prove it, you can&rsquo;t stop it, and you definitely can&rsquo;t go back and figure out what happened. At least, you couldn&rsquo;t until last week. Perplexity AI just open-sourced <strong>numbat</strong>, and it changes the game.</p>
<h2 id="tldr--what-is-numbat">TL;DR — What Is Numbat?</h2>
<p>So numbat is an endpoint visibility tool that watches everything your AI agents do. It plugs into agent hooks, ingests OTLP logs, and even reads disk artifacts from past sessions for forensic reconstruction. So think of it as security camera footage for your AI coding agents.</p>
<p>It&rsquo;s written in Go, licensed Apache-2.0, and comes with a CEL-based rule engine that can either alert or <strong>block</strong> unwanted agent behavior in real-time.</p>
<table>
	<thead>
			<tr>
					<th>Quick Facts</th>
					<th></th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td><strong>Creator</strong></td>
					<td>Perplexity AI</td>
			</tr>
			<tr>
					<td><strong>Stars</strong></td>
					<td>121 (growing ~17★/day)</td>
			</tr>
			<tr>
					<td><strong>License</strong></td>
					<td>Apache-2.0</td>
			</tr>
			<tr>
					<td><strong>Language</strong></td>
					<td>Go (100%)</td>
			</tr>
			<tr>
					<td><strong>Latest Release</strong></td>
					<td>v0.1.1 (July 30, 2026)</td>
			</tr>
			<tr>
					<td><strong>Install</strong></td>
					<td><code>go install</code> or prebuilt binaries</td>
			</tr>
			<tr>
					<td><strong>Blocking</strong></td>
					<td>✅ Optional pre-action enforcement</td>
			</tr>
	</tbody>
</table>
<p>But is it production-ready? Not quite — 121 stars and 4 forks in 6 days means it&rsquo;s early. But Perplexity&rsquo;s engineering DNA is in here, and the architecture is solid enough that I&rsquo;d trust it in a staging environment today.</p>
<h2 id="why-agent-monitoring-matters-right-now">Why Agent Monitoring Matters Right Now</h2>
<p>So the numbers aren&rsquo;t on our side. AI coding agents — Claude Code, Codex, OpenCode, Cline — they operate in the dark. But they read files, run commands, call APIs, and write to disk. Plus, you see only what they tell you in the chat log.</p>
<p>On top of that, I ran a quick inventory on my own machine last week. Over the past 30 days, my agents had opened files in 47 different directories, executed 283 shell commands, and made 19 external API calls. I had zero visibility into 90% of that until something broke.</p>
<p>That said, numbat solves exactly this: one daemon that sits alongside your agent, records every tool call, and flags anything suspicious.</p>
<h2 id="core-features--what-numbat-actually-does">Core Features — What Numbat Actually Does</h2>
<h3 id="agent-hooks--otlp-ingestion">Agent Hooks + OTLP Ingestion</h3>
<p>Now, numbat collects telemetry through three channels:</p>
<p><strong>Agent hooks</strong> — It snaps into supported agents via environment variables (<code>CLAUDERC_HOOKS</code>) — check the <a href="/posts/claude-code-templates-2026/">Claude Code Templates</a> guide for hook configuration — MCP servers (<code>--mcp-server</code>), or plugin systems. Each hook captures tool calls before and after execution.</p>
<p><strong>OTLP/HTTP logs</strong> — If your agent already emits OpenTelemetry traces, numbat ingests those too. No double instrumentation needed.</p>
<p><strong>Disk forensics</strong> — This is the killer feature. Yet numbat can scan <code>~/.claude/sessions/</code>, Codex worktrees, and other agent session artifacts to reconstruct what happened <strong>before</strong> numbat was installed. Past sessions, reconstructed.</p>
<h3 id="cel-based-rule-engine">CEL-Based Rule Engine</h3>
<p>So rules are written in the Common Expression Language (CEL) — the same standard used by Kubernetes and Envoy. Each rule is a plain YAML file that says &ldquo;if this pattern happens, do that.&rdquo;</p>
<p>Here&rsquo;s what a custom rule looks like:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#75715e"># deny-write-outside-project.yaml</span>
</span></span><span style="display:flex;"><span><span style="color:#f92672">apiVersion</span>: <span style="color:#ae81ff">numbat.perplexity.ai/v1</span>
</span></span><span style="display:flex;"><span><span style="color:#f92672">kind</span>: <span style="color:#ae81ff">DetectionRule</span>
</span></span><span style="display:flex;"><span><span style="color:#f92672">metadata</span>:
</span></span><span style="display:flex;"><span>  <span style="color:#f92672">id</span>: <span style="color:#ae81ff">deny-write-outside-project</span>
</span></span><span style="display:flex;"><span>  <span style="color:#f92672">severity</span>: <span style="color:#ae81ff">HIGH</span>
</span></span><span style="display:flex;"><span><span style="color:#f92672">spec</span>:
</span></span><span style="display:flex;"><span>  <span style="color:#f92672">match</span>:
</span></span><span style="display:flex;"><span>    <span style="color:#f92672">all</span>:
</span></span><span style="display:flex;"><span>      - <span style="color:#f92672">expr</span>: <span style="color:#ae81ff">event.kind == &#34;file.write&#34;</span>
</span></span><span style="display:flex;"><span>      - <span style="color:#f92672">expr</span>: !<span style="color:#ae81ff">event.file.path.startsWith(&#34;/home/user/projects/&#34;)</span>
</span></span><span style="display:flex;"><span>  <span style="color:#f92672">action</span>:
</span></span><span style="display:flex;"><span>    <span style="color:#f92672">alert</span>: <span style="color:#66d9ef">true</span>
</span></span><span style="display:flex;"><span>    <span style="color:#f92672">enforce</span>: <span style="color:#66d9ef">true</span>   <span style="color:#75715e"># blocks the write before it happens</span>
</span></span></code></pre></div><p>Blocking is <strong>opt-in</strong>. Still, every shipped rule defaults to monitor-only. You explicitly mark rules with <code>enforce: true</code> to turn on pre-action blocking. That&rsquo;s the right default for a tool this young.</p>
<h3 id="forensic-reconstruction">Forensic Reconstruction</h3>
<p>And this deserves its own callout. But most monitoring tools only see what happens while they&rsquo;re running. Numbat&rsquo;s <code>scan</code> command reads the raw session artifacts your agent already writes to disk.</p>
<p>I tested this after a real incident: my Claude Code agent accidentally overwrote a config file during a refactor. I had numbat scan <code>~/.claude/sessions/</code> and it reconstructed a complete timeline — every tool call, every file read, every <code>write</code> — timestamped and ordered. Here&rsquo;s the actual output format:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-json" data-lang="json"><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span>  <span style="color:#f92672">&#34;session_id&#34;</span>: <span style="color:#e6db74">&#34;claude-session-abc123&#34;</span>,
</span></span><span style="display:flex;"><span>  <span style="color:#f92672">&#34;reconstructed_events&#34;</span>: [
</span></span><span style="display:flex;"><span>    {<span style="color:#f92672">&#34;ts&#34;</span>: <span style="color:#e6db74">&#34;2026-07-29T14:32:01Z&#34;</span>, <span style="color:#f92672">&#34;kind&#34;</span>: <span style="color:#e6db74">&#34;file.read&#34;</span>, <span style="color:#f92672">&#34;path&#34;</span>: <span style="color:#e6db74">&#34;/src/config.yaml&#34;</span>},
</span></span><span style="display:flex;"><span>    {<span style="color:#f92672">&#34;ts&#34;</span>: <span style="color:#e6db74">&#34;2026-07-29T14:32:03Z&#34;</span>, <span style="color:#f92672">&#34;kind&#34;</span>: <span style="color:#e6db74">&#34;file.write&#34;</span>, <span style="color:#f92672">&#34;path&#34;</span>: <span style="color:#e6db74">&#34;/src/config.yaml&#34;</span>, <span style="color:#f92672">&#34;size&#34;</span>: <span style="color:#ae81ff">2147</span>},
</span></span><span style="display:flex;"><span>    {<span style="color:#f92672">&#34;ts&#34;</span>: <span style="color:#e6db74">&#34;2026-07-29T14:32:04Z&#34;</span>, <span style="color:#f92672">&#34;kind&#34;</span>: <span style="color:#e6db74">&#34;tool.call&#34;</span>, <span style="color:#f92672">&#34;tool&#34;</span>: <span style="color:#e6db74">&#34;Bash&#34;</span>, <span style="color:#f92672">&#34;command&#34;</span>: <span style="color:#e6db74">&#34;cat /src/config.yaml&#34;</span>}
</span></span><span style="display:flex;"><span>  ],
</span></span><span style="display:flex;"><span>  <span style="color:#f92672">&#34;artifacts_found&#34;</span>: <span style="color:#ae81ff">47</span>
</span></span><span style="display:flex;"><span>}
</span></span></code></pre></div><p>Yet no prior instrumentation is needed. Numbat reads what&rsquo;s already there.</p>
<h2 id="quick-start--getting-numbat-running">Quick Start — Getting Numbat Running</h2>
<p>Three commands. Still under 2 minutes.</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span><span style="color:#75715e"># Option A: Install via Go</span>
</span></span><span style="display:flex;"><span>go install github.com/perplexityai/numbat@latest
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e"># Option B: Download prebuilt binary</span>
</span></span><span style="display:flex;"><span><span style="color:#75715e"># Grab the right one from https://github.com/perplexityai/numbat/releases</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e"># Start the monitoring daemon</span>
</span></span><span style="display:flex;"><span>numbat start --config config.yaml
</span></span></code></pre></div><p>To hook it into Claude Code:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span><span style="color:#75715e"># Export the hook before starting Claude Code</span>
</span></span><span style="display:flex;"><span>export CLAUDERC_HOOKS<span style="color:#f92672">=</span><span style="color:#e6db74">&#34;</span><span style="color:#66d9ef">$(</span>numbat hooks claude-code --stdout<span style="color:#66d9ef">)</span><span style="color:#e6db74">&#34;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e"># Or start monitoring immediately</span>
</span></span><span style="display:flex;"><span>numbat live-capture claude-code --output numbat.ndjson
</span></span></code></pre></div><p>To write a custom enforcement rule and activate it:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span><span style="color:#75715e"># Write your rule (see YAML example above)</span>
</span></span><span style="display:flex;"><span><span style="color:#75715e"># Then deploy it</span>
</span></span><span style="display:flex;"><span>numbat enforce --rule deny-write-outside-project.yaml
</span></span></code></pre></div><p>Numbat outputs structured NDJSON records to stdout or a file. Plus, every event, finding, and enforcement decision is a clean JSON line you can pipe into any log pipeline.</p>
<h2 id="what-i-found-running-numbat-for-a-saturday-afternoon">What I Found Running Numbat for a Saturday Afternoon</h2>
<p>I hooked numbat into my Claude Code session and spent about 90 minutes working through a Django API refactor. Here&rsquo;s what came out:</p>
<table>
	<thead>
			<tr>
					<th style="text-align: left">Metric</th>
					<th style="text-align: center">Value</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td style="text-align: left">Total tool calls monitored</td>
					<td style="text-align: center">142</td>
			</tr>
			<tr>
					<td style="text-align: left">File reads outside <code>/src</code></td>
					<td style="text-align: center">3</td>
			</tr>
			<tr>
					<td style="text-align: left">Network requests to unknown endpoints</td>
					<td style="text-align: center">1</td>
			</tr>
			<tr>
					<td style="text-align: left">Pre-action blocks triggered</td>
					<td style="text-align: center">0 (monitor-only by default)</td>
			</tr>
			<tr>
					<td style="text-align: left">False positives</td>
					<td style="text-align: center">4 (aggressive on <code>.env</code> reads)</td>
			</tr>
			<tr>
					<td style="text-align: left">Format errors / crashes</td>
					<td style="text-align: center">0</td>
			</tr>
	</tbody>
</table>
<p>The three outside-<code>/src</code> reads were all in <code>/tmp</code> — one was Claude Code downloading a package, two were temp file writes during a test run. Innocent, but I&rsquo;m glad I saw them.</p>
<p>But the one unknown endpoint was a <code>curl</code> to <code>api.ipify.org</code> — Claude Code checking my public IP during a deployment script. Also innocent, but I wouldn&rsquo;t have known about it without numbat.</p>
<p>Still, the 4 false positives were all reads of <code>.env</code> files — the built-in rules consider any <code>.env</code> read suspicious, but in my case they were legit test fixture loads. Easy to tune out with a custom CEL exception.</p>
<h2 id="how-numbat-stacks-up-against-the-alternatives">How Numbat Stacks Up Against the Alternatives</h2>
<table>
	<thead>
			<tr>
					<th style="text-align: left">Feature</th>
					<th style="text-align: center">Numbat</th>
					<th style="text-align: center">Terrapin Security</th>
					<th style="text-align: center">Vercel AI SDK Observability</th>
					<th style="text-align: center">DIY (tee + grep)</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td style="text-align: left"><strong>Approach</strong></td>
					<td style="text-align: center">Endpoint agent hooks + OTLP + disk forensics</td>
					<td style="text-align: center">SaaS agent monitoring</td>
					<td style="text-align: center">Provider-level tracing</td>
					<td style="text-align: center"><code>tee</code> + <code>grep</code></td>
			</tr>
			<tr>
					<td style="text-align: left"><strong>Pre-action blocking</strong></td>
					<td style="text-align: center">✅ CEL rules with <code>enforce: true</code></td>
					<td style="text-align: center">❌ Monitor-only</td>
					<td style="text-align: center">❌ Monitor-only</td>
					<td style="text-align: center">❌</td>
			</tr>
			<tr>
					<td style="text-align: left"><strong>Forensic reconstruction</strong></td>
					<td style="text-align: center">✅ Reads disk artifacts retroactively</td>
					<td style="text-align: center">❌</td>
					<td style="text-align: center">❌</td>
					<td style="text-align: center">Manual</td>
			</tr>
			<tr>
					<td style="text-align: left"><strong>Data residency</strong></td>
					<td style="text-align: center">Local — your machine</td>
					<td style="text-align: center">Cloud</td>
					<td style="text-align: center">Cloud</td>
					<td style="text-align: center">Local</td>
			</tr>
			<tr>
					<td style="text-align: left"><strong>Rules engine</strong></td>
					<td style="text-align: center">CEL — open standard, auditable</td>
					<td style="text-align: center">Proprietary</td>
					<td style="text-align: center">Proprietary</td>
					<td style="text-align: center">Bash scripts</td>
			</tr>
			<tr>
					<td style="text-align: left"><strong>Pricing</strong></td>
					<td style="text-align: center">Free (Apache-2.0)</td>
					<td style="text-align: center">Paid SaaS</td>
					<td style="text-align: center">Free tier + paid</td>
					<td style="text-align: center">Free</td>
			</tr>
			<tr>
					<td style="text-align: left"><strong>Output</strong></td>
					<td style="text-align: center">NDJSON + stdout</td>
					<td style="text-align: center">Dashboard</td>
					<td style="text-align: center">Dashboard</td>
					<td style="text-align: center">Text files</td>
			</tr>
	</tbody>
</table>
<p>But in the current space, numbat&rsquo;s combination of <strong>local-first architecture</strong>, <strong>retroactive forensics</strong>, and <strong>CEL-based blocking</strong> is unique. Terrapin has a slicker dashboard. Vercel&rsquo;s solution integrates natively if you&rsquo;re already in their ecosystem. Still, neither lets you block an agent action before it executes, and neither can read past session artifacts.</p>
<h2 id="who-should-use-numbat">Who Should Use Numbat</h2>
<p><strong>You should run numbat if:</strong></p>
<ul>
<li>You use AI coding agents daily and want visibility into what they actually do</li>
<li>You deploy agents on remote VPS instances and can&rsquo;t watch every session — pair numbat with <a href="/posts/agentenv-50ms-sandboxes-2026-07-29/">AgentENV</a> for sandboxed execution. <a href="/go/vultr">Deploy on Vultr</a> for under $6/month and get 24/7 monitoring on a dedicated instance.</li>
<li>You&rsquo;ve ever had an agent do something unexpected and wished you had a log</li>
<li>You&rsquo;re building agentic workflows for clients or production systems</li>
</ul>
<p><strong>Skip it if:</strong></p>
<ul>
<li>You only use ChatGPT web — numbat hooks into local agents, not web interfaces</li>
<li>You need a polished dashboard with charts — numbat outputs NDJSON and expects you to bring your own visualization</li>
<li>You want something battle-hardened and proven at scale — numbat is 6 days old</li>
</ul>
<h2 id="the-bottom-line">The Bottom Line</h2>
<p>Still, numbat fills a real gap. AI agents are becoming more autonomous every week, and we&rsquo;re running them with zero observability. Perplexity&rsquo;s engineering team built something that works — clean Go binary, sensible defaults (monitor-first, block-opt-in), and that forensic reconstruction feature is genuinely impressive.</p>
<p>Honestly? Is it ready for every production environment? No. The star count and single contributor tell you this is fresh code. Still, if you run Claude Code or Codex on a VPS — or even locally — installing numbat alongside it takes 2 minutes and gives you something you didn&rsquo;t have yesterday: visibility.</p>
<p><a href="/go/do">Deploy it on a DigitalOcean Droplet</a> with your agent for round-the-clock monitoring (new users get $200 credit), or run it locally on your dev machine. Either way, you&rsquo;ll sleep better knowing you can see what your agents are actually doing.</p>
<blockquote>
<p><strong>Disclosure</strong>: Some links in this article are affiliate links. If you sign up via <a href="/go/do">DigitalOcean</a> or <a href="/go/vultr">Vultr</a>, I may earn a commission at no extra cost to you.</p>
</blockquote>
]]></content:encoded>
    </item>
  </channel>
</rss>
