What you get
Each LLM call records:- The model and provider.
- The prompts sent and the completions returned, with roles and tool calls.
- Prompt, completion, and total token counts, plus cached tokens where the provider reports them.
- Cost, calculated from the token counts.
- Errors, with the failure attached to the span that caused it.
Requirements
LLM monitoring needs a newer runtime than the rest of the agent:What you install
Two packages: the agent, and the LLM monitoring that goes with it:atatus-nodejs-llm brings its own instrumentation for every library in
Supported libraries, so there is nothing to add per library and nothing to
wire up.
Then enable LLM monitoring and restart your application. On startup the
agent logs what it instrumented:
atatus-nodejs-llm is a large install, because it carries the SDKs its instrumentation is built
against. Install it only on applications that make LLM calls. Applications that do not can install
atatus-nodejs on its own, exactly as before.Enable LLM monitoring
Set two options when you start the agent:require('atatus-nodejs').start(...) as the first line of your entry file, or preload it
with node -r atatus-nodejs/start app.js.
On startup the agent logs which libraries it instrumented, so you can confirm the ones you use were
picked up.
Supported libraries
These are traced automatically. You do not install anything per library. A library you use is traced, one you do not is ignored, and the agent carries on either way.
Two more are supported but work differently, and are covered below: the
Vercel AI SDK and TanStack AI.
Library versions
Instrumentation supports a range of versions of each library, and a library outside that range is left alone. When that happens the agent says so at startup, naming the library, the version you have, and the versions it can trace:Frameworks that call another traced library
A framework and the provider SDK underneath it are traced independently, so one model call made through a framework produces two LLM spans: one for the framework step and one for the provider call it made. LangChain calling OpenAI is the common case. Both spans are kept, with their prompts, completions, model and timing. Token counts and cost are recorded on the framework span only, so a call is never counted twice. The provider span shows zero tokens for that reason, and says so in its attributes. You do not need to configure anything for this. If you would rather not see the second span at all, exclude the provider, but only if you never call it directly:The HTTP request behind a model call
A model call already records the provider, the model, the timing and the outcome, so the outgoing HTTP request underneath it would be the same event a second time. That span is not reported. It is reported when the request failed. Nothing else records a failed model call, because the instrumentation does not produce an LLM span when a call is rejected. On a rate limit or a server error you still see the outgoing request, with its status andfailure outcome, in APM.
Outgoing requests that are not part of a model call are unaffected.
Choose which libraries to trace
By default every library above that you have installed is traced. To narrow that down, list the ones you want:Vercel AI SDK
The Vercel AI SDK reports its own telemetry rather than being instrumented, so there is nothing extra to install. Turn on its telemetry per call:TanStack AI
TanStack AI is traced through a middleware you attach yourself. It ships withatatus-nodejs-llm, so there is nothing extra to install:
llmobs: true.
This middleware is published as an ES module. Import it with
import, not require.Trace your own agent code
The libraries above are traced on their own. Code they cannot see, such as an agent loop you wrote yourself, a pipeline step, or a tool your app calls directly, shows up only as the model calls made inside it. The trace then reads as a flat list of LLM calls with no agent or tool around them. Wrap that code with the helpers fromatatus-nodejs-llm to give the trace its shape:
The options are:
Pass
processInput or processOutput when an argument or result is large or sensitive, and return
only what should be recorded:
this, return value and errors. A throw or a rejected
promise marks the span as failed and is passed on unchanged.
You can wrap functions at module load, before or after the agent starts. With LLM monitoring off, the
wrapped function runs normally and records nothing.
ES modules
Automatically traced libraries work under ES modules. Start your application with the agent’s loader:--experimental-loader=atatus-nodejs/loader.mjs works too, on Node versions that still
accept it. Nothing else to configure.
You may see one line like
import-in-the-middle failed to wrap ...index-incubating.js at startup.
It is harmless: that one file cannot be rewritten for tracing, and everything else is traced
normally. Nothing is lost.Configuration reference
Turning on
llmobs also turns on the agent’s OpenTelemetry bridge, which is how LLM spans are
collected. You do not need to set that yourself.
Prompt and completion length
Prompts and completions are sent up tolongFieldMaxLength, which defaults to 10,000 characters.
Anything longer is cut at that point. Raise it if you send large prompts and want to see them whole:
Where the data appears
Your application shows up under both APM and LLM in the dashboard. The APM views are unchanged, and the LLM views add the model, token, cost, and prompt detail. An application appears under LLM only after it has sent its first LLM span, so make a call through one of the traced libraries before looking for it.Next steps
- Span kinds explains how each kind of LLM span is classified.
- Cost and tokens covers how cost is calculated.
- Troubleshooting helps if no data arrives.