Alert Threshold

A threshold defines the boundary between normal and abnormal metric behavior. When a metric crosses the threshold for the configured duration, Atatus creates an issue and notifies you.

Threshold levels

Atatus supports two threshold levels per rule — Warning and Critical — each evaluated independently.

Both levels can have different threshold values and are evaluated on every scheduler tick. For example, you might set Warning at 70% CPU and Critical at 90% CPU.

If only a Critical threshold is configured, no Warning issues are created. The Warning threshold is optional.

Evaluation windows

The evaluation window (duration) controls how long a metric must breach the threshold before an alert triggers. Available windows:

Longer durations reduce false positives but delay detection. Shorter durations catch issues faster but may trigger on transient spikes.

Time functions

The time function controls how violations within the evaluation window are counted.

Example: A 10-minute window with all requires 10 consecutive minutes of violation. The same window with any triggers on the first breaching minute.

Operators

Most alerts use above (e.g., response time above 2 seconds, CPU above 90%). Use below for metrics where a drop is concerning (e.g., throughput drops below 100 requests/minute, replicas available drops below desired count).

No-data handling

If there are not enough data points within the evaluation window, the alert returns an indeterminate result and does not trigger. This prevents false positives when metrics are sparse or temporarily missing.

For infrastructure monitoring, the Host Not Reporting rule type specifically detects missing data — it triggers when a host stops sending metrics for the configured duration.

How to choose threshold values

1. Start generous, tighten over time. Set initial thresholds higher than you think necessary, then lower them as you learn normal operating ranges.
2. Use Warning for early awareness, Critical for action. Warning thresholds give you time to investigate before a situation becomes urgent.
3. Check historical data. Look at your metric's normal range over the past week before setting thresholds. Set Warning at the upper end of normal and Critical well beyond it.
4. Match duration to the metric's behavior. Metrics with natural variability (CPU, network throughput) need longer durations (15-30 min). Stable metrics (error rate, availability) can use shorter durations (5 min).
5. Prefer all over any for most metrics. The all time function prevents false positives from momentary spikes. Use any only for metrics where a single breach is genuinely critical.

Examples

APM response time

Alert when web response time exceeds 2 seconds for 5 consecutive minutes.

Infrastructure CPU

Alert when CPU usage exceeds 90% at any point in a 15-minute window.

Kubernetes replicas

Alert when available replicas drops below the desired count for 5 minutes.