The AI Helper now shows exactly how much of your prompt hits the cache
Up to v3.2.70 the context meter in the AI Helper estimated token usage by the rule "characters / 4". It worked, but it was off by 10–30% depending on the model and language. In v3.2.71–73 we switched to real numbers from the API — with an extra breakdown of how many tokens land in the cache (which costs ~10× less than fresh input).
What prompt cache is
All three of the big LLM APIs (Anthropic Claude, OpenAI GPT, Google Gemini) support caching long prompt prefixes. If you keep sending requests with the same start (the same system prompt + DOM snapshot + conversation history), the second and later calls don't pay for those tokens again — the provider recognizes the same prefix and charges you only the cache-hit price (~10% of full token price).
For the AI Helper, which works through a conversation step by step — the system prompt doesn't change, the DOM tree changes rarely, the history grows linearly — the cache typically catches 70–95% of the prompt. That's the difference between a dollar and ten cents per round.
What you now see in the AI Helper (v3.2.73)
Below the chat window, in the stats bar, there are three separate token indicators:
- Fresh input — the tokens of this turn that the API had to process from scratch (your new message + any new DOM).
- Cached — the tokens recognized as a prefix of the previous turn — they cost a fraction of the normal price.
- Output — what the model generated (always full price, never cached).
Sum of input + cached is the total length of your prompt for this turn — the equivalent of the old single "input". Disjoint — no double counting.
Where the numbers come from
Providers return usage in API response fields. Each has its own naming:
- Anthropic (Claude) —
usage.input_tokens,usage.cache_creation_input_tokens,usage.cache_read_input_tokens,usage.output_tokens. - OpenAI (GPT-4o, o1) —
usage.prompt_tokens,usage.prompt_tokens_details.cached_tokens,usage.completion_tokens. - Google (Gemini) —
usageMetadata.promptTokenCount,usageMetadata.cachedContentTokenCount,usageMetadata.candidatesTokenCount.
The AI Helper reads each provider's response and normalizes it into a uniform format (input / cached / output). The indicator updates immediately after the turn ends — no extra calls, no estimation.
What it gives you in practice
Three concrete things you didn't know before v3.2.73:
- Real cost per round — you have the exact numbers to multiply by your provider's rate. No online calculators, no guesses. What's billed = what's shown.
- Cache hit ratio — when you start a new conversation, cached = 0 (the cache is being built). In subsequent turns you see it climb. A cache hit of 90%+ means the conversation is running efficiently; 30% means you're still throwing in large new context.
- "Keep going or start fresh" — when you see cached starting to drop (e.g. the DOM tree changed and the prefix doesn't match anymore), you know it's easier to restart the conversation with new context than to throw more turns at it.
Why this matters
LLM APIs are cheaper than they were a year ago, but cents still add up under heavy use. We use the AI Helper to build complex CSS/JS rules across dozens of domains — each round is 5–20 thousand tokens. Without cache, that's dollars per month; with a 90% cache hit, tens of cents. Showing real numbers helps shape the workflow that way.
Provider notes
Cache is provider-specific — different TTL, different minimum prefix size, different prices. For orientation:
- Anthropic Claude — cache TTL 5 minutes, minimum 1024 tokens in prefix, cache write = 1.25× input price, cache read = 0.1× input price.
- OpenAI GPT-4o / o1 — automatic cache from 1024 tokens, no separate write fee, cache read = 0.5× input price.
- Google Gemini 1.5 — explicit context caching API, the user picks what to cache, write + storage fee, read cheaper than fresh input.
Values current at the time of writing (May 2026) — check live rates with your provider.
See also
- On-page windows — the AI Helper as one of the developer windows
- AI Helper in a shadow DOM — a recent step in the window's style isolation
- Interactive onboarding tutorial — from zero to a working rule with the AI Helper
Install JustZix — and use real numbers instead of estimates.
Rate this post
No ratings yet — be the first.