Custom rules let you detect values that the library rules do not cover, such as an internal account number, a partner API key, or a national identifier for a country you operate in.

This page covers writing the pattern, testing it, and the practices that keep custom rules from causing trouble.

When to write a custom rule

Write one when the value you care about has a shape that is specific to you:

  • Internal identifiers, for example CUST-4471902.
  • API keys or tokens issued by your own services.
  • Region specific identifiers not in the library.
  • Any field your compliance program names explicitly.

If a library rule already covers the format, use it. Library patterns are maintained and tuned by Atatus, and you will not have to revisit them.

Create a custom rule

  1. Open a scanning group and select Add Rule.
  2. Choose Custom Rule.
  3. Fill in Rule Name and, optionally, a description.
  4. Enter your regular expression in Define Regex Pattern. The form flags Invalid regex pattern if it will not compile.
  5. Add proximity keywords if the pattern alone is ambiguous.
  6. Choose the scope and the match action.
  7. Add at least one tag in key:value form. Tags are required for custom rules.
  8. Save the rule, then test it and enable it.

Writing the pattern

Keep patterns simple and specific. The scanning engine runs your expression against every event the group sees, so a broad pattern costs you both accuracy and throughput.

Supported syntax

Stick to the constructs that every regular expression engine agrees on:

  • Character classes: [A-Z], \d, \w, \s
  • Quantifiers: {4}, {3,6}, +, *, ?
  • Word boundaries: \b
  • Alternation and grouping: (abc|def)
  • Non-capturing groups: (?:...)
Warning:

Avoid backreferences and lookahead or lookbehind assertions. The form validates your pattern in the browser, which is more permissive than the engine that runs in the ingest pipeline. A pattern can pass validation in the form and still fail to compile when it runs. Always confirm with the evaluator, which runs the real engine.

Anchor with word boundaries

Wrap the pattern in \b so it does not match a fragment of a longer string:

\bCUST-\d{7}\b

Without the boundaries, the same pattern matches the CUST-4471902 inside XCUST-44719021, which is probably not what you want.

Avoid greedy wildcards

A pattern such as key=.* matches to the end of the line and will redact far more than the key. Match the shape of the value itself instead:

\bak_live_[A-Za-z0-9]{24}\b

Worked examples

Internal customer ID

For IDs like CUST-4471902:

\bCUST-\d{7}\b

The prefix makes this unambiguous, so no proximity keywords are needed. A partial redaction that keeps the last three digits works well here, since support staff can still correlate a ticket without the full ID.

Internal API key

For keys like ak_live_9f2c1d7b4e8a03c5d6b1f7e2:

\bak_(live|test)_[A-Za-z0-9]{24}\b

Use the Redact action with replacement text of [api_key], and tag it sensitive_data:internal_api_key and sensitive_data_category:credentials.

Employee number that needs keywords

For a bare six digit employee number, the pattern alone would match almost any number in your logs:

\b\d{6}\b

Add proximity keywords such as employee, emp, staff id, and badge, and leave Characters before match at the default of 30. Now the rule only fires when one of those words sits just before the number.

Tag format

Custom rules require at least one tag, and tags must be key:value pairs containing only letters, numbers, and underscores. Hyphens, spaces, and other punctuation are rejected with Invalid tag format.

Follow the same convention as the library rules so your findings sit alongside everything else:

sensitive_data:internal_customer_id
sensitive_data_category:pii

Test before you enable

Save the rule, then reopen it to edit. The evaluator appears in the edit form. Paste in a realistic sample and check the highlighted matches and the match count.

Test with three kinds of input:

  1. A value you expect to match, to confirm the pattern works.
  2. A near miss that should not match, such as an order number of the same length.
  3. A real log line from your application, since the surrounding text is what proximity keywords act on.

Practices worth following

  • Start with No Action. Let the rule run for a day with tags only, review what it matched, then switch to a masking action.
  • Scope narrowly when you can. If the value only ever appears in one attribute, use Specific event scoping and name that attribute. It is faster and safer than scanning the whole event.
  • Prefer a prefix over a length. \bCUST-\d{7}\b is far more reliable than \b\d{7}\b with keywords.
  • One format per rule. If a value arrives in two formats, write two rules. Combining them with alternation makes both harder to tune.
  • Review after a rollout. New services log new things. Check your findings periodically to catch patterns that have drifted.
Warning:

A custom rule with a masking action permanently changes stored data. There is no undo and no way to recover the original value. Test with the evaluator first.

Next steps