Alert Thresholds That Reduce False Positive Noise

Alert Thresholds That Reduce False Positive Noise

Setting an alert threshold too tight buries a security team in noise; set it too loose and a genuine subdomain takeover slips through unnoticed for weeks. Getting DNS monitoring alert thresholds right is less about picking a magic number and more about understanding what normal DNS behavior looks like for a specific domain portfolio before deciding what counts as abnormal.

Why default thresholds rarely fit real environments

Most DNS monitoring tools ship with conservative defaults designed to avoid missing anything – which means they flag almost everything. A TTL change from 3600 to 300 seconds, a new NS record during a registrar migration, a CNAME that briefly resolves to NXDOMAIN during a CDN failover – all of these can trigger an alert under out-of-the-box settings.

The problem compounds with scale. An organization running 40 subdomains might tolerate a dozen alerts a week. One running 400 subdomains across marketing, engineering, and partner integrations will drown in hundreds of notifications, and within a month the on-call rotation starts skimming subject lines instead of reading payloads. That’s the exact failure mode that let the 2016 Democratic National Committee-adjacent typosquatting campaigns and countless smaller subdomain takeovers go unnoticed – not a lack of alerts, but too many low-value ones training staff to ignore the channel.

Separate alert categories by actual risk, not by record type

A common mistake is building alert rules around DNS record types (all CNAME changes, all MX changes) rather than around business risk. Not every CNAME change matters equally. A CNAME pointing to a decommissioned Heroku app or an expired S3 bucket name is a takeover risk requiring immediate action. A CNAME update reflecting a planned migration from one Cloudflare zone to another, announced in the change management ticket three days earlier, is noise.

Group alerts into three tiers instead:

Critical – dangling CNAME/NS records pointing to unclaimed or deprovisioned cloud resources, sudden authoritative nameserver changes outside a known maintenance window, and SPF/DKIM/DMARC record deletions on domains that send production mail.
Warning – TTL drops below 300 seconds without a corresponding change ticket, new subdomains appearing that weren’t part of the last discovery scan, and wildcard record additions.
Informational – routine TTL increases, IP address rotation within a known CDN’s published ranges, and A record changes on subdomains already tagged as actively managed.

Only the critical tier should page someone at 2 a.m. Warning-tier items belong in a daily digest. Informational items should be logged but not surfaced unless someone is specifically auditing that record.

Building a baseline before touching threshold settings

An experienced DNS administrator doesn’t start by adjusting sensitivity sliders. They pull two to four weeks of historical change data first – which records change, how often, and during which windows – before deciding what deviates from that pattern. Without that baseline, threshold tuning is guesswork dressed up as configuration.

For a mid-size SaaS company with roughly 150 subdomains, a realistic baseline often looks like this: 5–10 legitimate DNS changes per week tied to deploys, 1–2 TTL adjustments during CDN vendor changes, and near-zero NS record changes outside of a registrar migration. Once that pattern is established, an alert firing on the fourth NS change in a single day is meaningful. The same alert on week one, before any baseline exists, is just noise with no context to judge it against.

This is also where keeping an accurate subdomain inventory pays off directly – a system can’t tell a new, potentially unauthorized subdomain from a planned addition if there’s no current record of what’s supposed to exist.

Tuning by asset criticality, not by uniform policy

Applying one threshold policy across an entire domain portfolio ignores the fact that not all subdomains carry equal risk. A forgotten marketing landing page (campaign.example.com) sitting dormant since a Q2 2024 product launch deserves different handling than checkout.example.com or api.example.com.

A practical approach tiers thresholds by asset function:

Production and payment-adjacent subdomains get the tightest thresholds – any unexpected change triggers immediate critical alerts, no exceptions, because the cost of a missed takeover on a checkout flow is measured in fraud and chargebacks, not inconvenience.

Marketing and campaign subdomains, which is where a large share of real-world takeovers actually happen because they’re built fast and abandoned faster, get moderate thresholds tuned to catch dangling records specifically – this is a case worth reading about in more detail in DNS security gaps in marketing and campaign subdomains.

Internal or test subdomains get loose thresholds focused only on takeover risk indicators (NXDOMAIN responses, unclaimed cloud resource pointers), since day-to-day churn there is expected and not worth flagging.

Common mistakes that keep noise levels high

Three patterns show up repeatedly in DNS monitoring setups that never get quiet:

Treating every alert as equally urgent instead of building the tiered system above – teams that do this eventually mute the whole channel, which is worse than having no monitoring at all because it creates false confidence.

Tuning thresholds once during initial setup and never revisiting them. A domain portfolio that added 60 subdomains after an acquisition needs a fresh baseline, not the settings configured for the pre-acquisition footprint.

Trusting a change as benign because it matches a “known” IP range or ASN without verifying the record actually points to infrastructure the organization still controls – attackers have registered abandoned cloud resources that still resolve within a legitimate-looking IP block, and a threshold rule that only checks “is this IP in our approved CIDR list” misses that entirely.

Frequently asked questions

How often should alert thresholds be reviewed?
Quarterly at minimum, and immediately after any infrastructure event – a cloud migration, an acquisition, a CDN vendor switch, or a period of rapid subdomain growth. A threshold tuned for 100 subdomains stops making sense once that number doubles.

What’s a reasonable false-positive rate to aim for?
Security teams that have tuned thresholds well typically see fewer than 5% of critical-tier alerts turn out to be non-issues. If critical alerts are being dismissed more than one in twenty times, the tiering criteria need adjustment, not the on-call team’s attention span.

Does tightening thresholds risk missing real incidents?
Only if tightening means suppressing alert categories rather than reclassifying them. The goal isn’t fewer alerts overall – it’s routing low-risk changes to a digest and reserving immediate notification for changes that match known takeover patterns, like a CNAME resolving to NXDOMAIN on a cloud provider known for reusable subdomain names.

Threshold tuning is never a one-time task. It’s a recurring discipline built on an accurate inventory, a real baseline of normal change activity, and a tiering system that matches alert urgency to actual business risk rather than record type. Start by measuring what “normal” looks like for the current domain portfolio – everything else follows from that.