The Sensitive Data Classifier (SDC) scans your logs as they arrive and acts on values that should not be stored in plain text, such as credit card numbers, email addresses, and authentication tokens. It runs inside the Atatus ingest pipeline, so matched values are masked before anything is written to storage.

This page explains the moving parts. If you want to get scanning right away, start with Set Up the Sensitive Data Classifier.

How scanning works

Scanning happens in the ingest path, before storage:

  1. Your agent sends logs. Log events arrive at the Atatus ingest endpoint as usual.
  2. Scanning groups decide what gets scanned. Each group can narrow the traffic with a filter query. Groups are evaluated in the order you arrange them.
  3. Rules look for matches. Every rule in the group runs its regular expression against the event, optionally checking for nearby keywords to cut down on false positives.
  4. The match action is applied. A matched value is hashed, redacted, partially redacted, or left alone, depending on the rule.
  5. Tags are attached. The event is tagged with the rule's sensitive_data and sensitive_data_category tags, so you can search for it later without ever seeing the original value.
  6. The event is stored. Only the masked version is written.
Warning:

Masking is permanent. Atatus never stores the original value, so a redacted or hashed value cannot be recovered later. Test a rule with the rule evaluator before you enable it on live traffic.

Key concepts

Concept What it is
Scanning group A container that decides what gets scanned: which subset of your logs, and at what sampling rate. Rules live inside a group.
Scanning rule A single pattern that decides what to look for and what to do when it matches.
Library rule A built-in rule maintained by Atatus. There are 35 of them, covering payment cards, PII, credentials, and network data. See Library Rules.
Custom rule A rule you write yourself with your own regular expression. See Custom Rules.
Match action What happens to a matched value: Hash, Redact, Partially Redact, or No Action.
Tags Key and value labels added to any event that matched, in the form sensitive_data:email_address. These drive search and alerting.

What you can scan

The Sensitive Data Classifier currently scans Logs, covering log messages and their attributes.

Note:

Support for scanning APM spans and API analytics is planned. This page will be updated when those products are available.

For Logs you choose a scanning method:

  • 100% scans every event in the group.
  • Sample scans a percentage of events, between 10% and 90%. Useful while you are tuning a new rule and want to limit the impact.
  • None leaves Logs out of the group, so the group scans nothing.

See Scanning Groups for the full configuration reference.

Match actions

Every rule ends in one of four actions:

Action Result
Hash Replaces the value with a non-reversible hash. The same input always produces the same hash, so you can still correlate events without seeing the value.
Redact Replaces the value with fixed text of your choice, for example [REDACTED].
Partially Redact Keeps part of the value and masks the rest. You choose how many characters to mask (1 to 16) and whether to mask from the left or the right. Useful for keeping the last four digits of a card.
No Action Leaves the value untouched but still tags the event. Use this to find out where sensitive data is flowing before you commit to masking it.
Tip:

Starting with No Action is a good way to roll out SDC. You get the tags and can see exactly what would be masked, without changing any stored data. Switch to Redact or Hash once the rule looks right.

Where to find it

The Sensitive Data Classifier lives under Security in the Atatus dashboard.

Next steps