IntelliJev

Find the right context. Review the right change.

@Taanviir · @aikram42 · @mahahahad

JetBrains Help the Developer hackathon

IntelliJev: one IntelliJ plugin that finds the files a task needs and lets the developer review a coding model's changes before applying them.
The problem

An agent's first minutes are a scavenger hunt

grep "retry"read Client.ktgrep "backoff" read Config.ktglob **/*Retry*read RetryPolicy.kt grep "delay"read ClientTest.ktedit

Each hop is a full LLM round-trip. On a 2,000-file project the agent reads files one at a time to find the few that matter, while the IDE already knows every file.

In our 8-task Koog pilot, Claude Code averaged 13.0 turns per task.

Chips illustrate a typical exploration. Turns: eval/results/claude_ab_sonnet_o40_t8.json (no-hook arm).
The problem: coding agents spend their first minutes re-discovering the project through a keyhole, one tool call at a time.
What IntelliJev is

One IntelliJ plugin: find the right context, review the right change

Find context

Rank the files a task needs

Jev in the cloud, local Laya, or fast keywords. Each provider keeps its own settings.

Agent access

pack_context

The same ranking over IntelliJ's MCP server and a Claude Code hook that runs before every prompt.

Review changes

Approve every edit

Selected files go to a coding model; you inspect the diff and apply only what you approve.

Jev, TypeSafe's decision model, judges and never writes: it answers one typed question about thousands of files, fast and cheaply. The looking goes to Jev; the writing stays with the LLM.

One installable zip: context engine and reviewed edits in the same tool window.
An LLM is the wrong tool for asking the same question 2,000 times. Jev is built for exactly that, so IntelliJev lets it do the looking and hands the result to the agent or to the review flow.
How it works

From every file to the right one in one pack

Every source file, sketched to ~300 tokens1.4 s
2,206files
Pass 1: Jev reads sketches · BM25 ranks full textin parallel
60sketches / call
Pool: top 60 from each
60 + 60pool
Pass 2: Jev re-reads the pool's full source, fused with BM25
6files / call
Stage 3: one comparison · which file must change?
top 10+0.4 s
#1 RetryingLLMClient.kt0.97"Add exponential backoff to retries"
4.4 s

per warm pack, 2,206 files

54–56

Jev calls per pack

$0.03

API fee per pack, about

Bar widths are not to scale. Pipeline and pack figures: plugins/context-packer/README.md, spike/RESULTS.md (Koog, 2,206 files). The #1 hit is one rehearsal pack on 1,705 .kt files (0.97 score), not a benchmark.
Jev alone on sketches loses to keyword search. The win comes from re-ranking the full source of a small pool and fusing with BM25; the final comparison sharpens the top five.
Result · retrieval

Finds the right files far more often

BM25 keyword searchIntelliJev (Jev pipeline)
0.42
0.57
0.53
0.69
0.63
0.80
recall@5
recall@10
recall@20
+16 pts

recall@10 over BM25, 95% interval +10 to +23

+0.03

recall@5 from stage 3 (0.54 → 0.57), interval +0.006 to +0.07

New repo · JetBrains/Exposed · 40 tasks0.43 → 0.54

recall@5, frozen settings, +0.10 [+0.01, +0.20]; recall@10 0.56 → 0.62, not significant

Local Laya · 30 Koog tasks$0

API fee; keyword search still won locally, recall@10 0.553 vs 0.391

70 held-out Koog commits (answer = files changed; tuned on 40 dev tasks, measured once). Exposed: 1,230 Jev calls, 0 errors. Laya: different tasks and machine, not vs Jev. Sources: spike/RESULTS.md, stage3_test_k10_c6000.json, second_repo_exposed_t40.json, eval/LAYA.md.
Real commits from JetBrains' own agent framework. Jev on sketches alone lost to BM25 (0.37 vs 0.46 in the spike), which is why it re-ranks full source. Nothing was re-tuned for Exposed. Laya is the offline option and keyword search beat it, so Fast keywords ships too.
Before / after

Same task, same agent. Fewer detours.

Without IntelliJev

The agent greps around

grepgrepreadgrep grepreadglobgrep readgrepgrepreadedit
13.0
turns / task
7.9
grep + glob / task
$2.21
spend, 8 tasks
With IntelliJev

It starts at the right file

hook: pack_contextread RetryingLLMClient.ktgrepgrep grepreadreadedit
9.8
turns · −25%
4.9
grep + glob · −38%
$1.88
spend · −15%
Retrieval, 70 held-out tasks: recall@10 0.53 → 0.69
8-task pilot, not a benchmark: Claude Code (Sonnet) with vs without the hook; median wall time unchanged at 26 s; recall equal on 7 of 8 tasks. Chips mirror the per-task means (7.9 searches, 4.1 reads without; 4.9 and 3.9 with); the RetryingLLMClient.kt start is one rehearsal run. Sources: eval/results/claude_ab_sonnet_o40_t8.json, spike/RESULTS.md.
The hook hands Claude ranked files before it starts. In the pilot that cut a quarter of the turns and over a third of the searches. It did not make the agent faster, and we say so.
Live demo

Watch the agent skip the hunt

"Add exponential backoff to retries in the LLM client"→ RetryingLLMClient.kt · Jev top pick 0.96 (rehearsal)
Also verified · keyword ranking put the right file first
"Support reasoning_content in Delta for OpenAI streaming responses"→ OpenAILLMClient.kt
"Map cachedContentTokenCount from Google usage metadata into token usage"→ GoogleLLMClient.kt
"Add a Spring Boot property to configure the Ollama base URL"→ OllamaKoogProperties.kt
"Add token counting for Anthropic prompts in the tokenizer"→ PromptTokenizer.kt
"Add streaming support to the A2A server"→ A2AServer.kt
Harder · keywords miss these, compare with Jev live
"Add a configurable timeout to MCP tool calls made by agents"keywords vs Jev
"Handle rate limit 429 errors from Bedrock with a retry"keywords vs Jev
Agents

Claude Code and Codex both get packed context through the same prompt hook. Codex run today: ranked the project in 5.5 s.

Single rehearsal checks on JetBrains/koog, not benchmarks. Flow: Find context → Pack → Review changes, or run the agent in the Koog folder.
Start with the main prompt in the IDE, then show the agent. If there's time, run one of the harder prompts with keywords and then Jev to show the difference. Backup: the recorded terminal run.
IntelliJev

Less looking. More doing. Thank you.

pitch reports site github.com/Taanviir/hackathon-jetbrains-202609
@Taanviir · @aikram42 · @mahahahad · Questions?
Close: Jev decides where to look, a coding model proposes the change, and the developer approves the edit. Everything, including the evals, is open.
← → navigate · N notes · F fullscreen