Reading Verdicts

A verdict is one judgement, made up of an evaluator, a label, a score, and optionally the judge’s reasoning. It renders as a colored chip beside the span, trace or session it judged.

​

Where chips appear

Verdicts show up on four surfaces: A chip reads Evaluator name: label. A cell shows at most three chips before collapsing the rest behind +N more, so a row judged by ten evaluators does not tower over its neighbors.

​

Colors and ordering

Colors are not stored. They are derived when the chip renders, from the label’s position in your list of choices, how many choices there are, and the evaluator’s optimization direction: Direction also sets the order chips appear in. Worst comes first, so the thing you need to look at is the first thing in the cell.

​

Why a chip says a count

Span-scope verdicts are collapsed per evaluator and label on a trace-grained row. A count of 3 means three spans of that trace got the same label from that evaluator. Open the trace to see which spans.

​

Why a session chip is on only one row

A session verdict is anchored to the session’s first trace, because that is the single row it can render on in a trace-grained list. Filtering a session verdict on the Traces page therefore returns that one row, not every trace in the session.

​

Explanations

Explanations is off by default. When you turn it on, the judge is also required to return its reasoning, and that reasoning is shown in three places: It costs extra completion tokens on every judged unit, and makes each call a little slower. Prompt, completion and total tokens are stored per verdict, so the difference is measurable.

​

Filtering and faceting by verdict

Every evaluator that produced a verdict in the current time window contributes a facet to the filter sidebar on Traces, Sessions and Explore, listing the labels it actually returned. Tick a label to filter, or use the negated form to exclude it. Multiple selections combine like this:

​

The grain follows the page, not the evaluator

A verdict filter returns rows in the unit the page shows, which is not always the unit that was judged: So a span-scope verdict ticked in all-spans mode is broader than the verdict itself, because it gives you the spans of the traces that contain a match. That is deliberate, since it keeps the facet from ever offering a value the table cannot show. Facet counts are always in the unit the list shows, so they count traces on Traces and Explore, sessions on Sessions, and spans in all-spans mode.

​

One chart limitation

A verdict filter cannot be served by the pre-aggregated time series chart, which carries no trace identity. Applying one falls back to the unaggregated query, which is the same behavior as filtering by status.

​

What is not shown, and why

Verdicts are stored only for clean successes. Everything else is deliberately invisible in the product, because a broken judge is an operational fact rather than a judgement about your trace:

​

How long verdicts last

A verdict inherits the retention of the span it judged, so chips age out exactly when the trace underneath them does. There is nothing separate to manage. Deleting an evaluator does not delete its verdicts. Each row carries the evaluator’s name, category, direction and color position, so existing chips keep rendering correctly until they age out. It also means two evaluators can judge the same trace and both chips appear, one per evaluator.

​

Re-judging with a new prompt

Editing an active evaluator’s prompt does not re-judge anything. Units already judged keep their old verdict, and new units get the new prompt. To re-score a range you have already judged, queue a backfill over it, and the newer verdict supersedes the older one for the same unit.