The common failure is not missing logs - it is having every log and no working detections. A SIEM with vendor default rules produces an alert volume no team can triage, so the queue is ignored, and the one alert that mattered is ignored with it.
Detections are code, not settings
The shift that makes this work is treating each detection as an engineered artefact: written deliberately, version-controlled, peer-reviewed, tested against known-good and known-bad data, and retired when it stops earning its place.
That means a detection has an owner, a documented intent, a stated data source, an expected false-positive profile, and a response action. A rule nobody can explain is a rule nobody will action.
Start from behaviour, not from tools
The productive order is: pick a technique an adversary would plausibly use against you, confirm you have the telemetry to see it, write the detection, test it, then measure. Working the other way - enabling everything your tool ships with - produces coverage that looks broad and is meaningless.
MITRE ATT&CK is useful here as a coverage map, with one caveat: counting techniques you have "covered" is a vanity metric. One tested, tuned detection for a technique your adversaries actually use beats forty untested rules across the matrix.
Connect detection coverage to control requirements
GRC Copilot maps monitoring and logging evidence to the controls each framework requires, so detection work counts toward compliance too.
Try GRC Copilot free Generate an AI-powered assessment Download checklist Book a demo
The false positive problem
False positives are not an annoyance, they are the failure mode. A detection firing fifty times a day with no true positives trains your analysts to dismiss it - and they will dismiss the fifty-first.
What helps:
- Tune before deploying against historical data - if it would have fired 400 times last month, it is not ready.
- Enrich rather than filter. Adding asset criticality, user role and geography turns a noisy rule into a precise one.
- Set a retirement threshold. Any detection with no true positive over a defined period gets reviewed or removed.
- Separate signal tiers. Not everything needs to page someone; much of it belongs in a hunt queue or a dashboard.
Testing
Detections that have never been triggered deliberately are assumptions. Validate by simulating the behaviour - atomic tests, purple team exercises, or replaying recorded attack telemetry - and confirm the alert fires, routes correctly and contains what the responder needs.
This is also where you find the unglamorous breakages: a log source that stopped shipping three weeks ago, a field that changed name after a platform upgrade, an alert routing to a distribution list nobody reads.
Metrics that mean something
- Alert-to-incident ratio per detection - the honest measure of quality.
- Mean time to detect for validated true positives.
- Log source health - are the feeds you depend on actually arriving?
- Detections retired, which is a sign of a healthy programme rather than a failing one.
Alert count is not a metric. Neither is number of rules enabled.
Frequently asked questions
Do we need a SIEM to start?
You need centralised logs and a way to query them. The product matters less than whether the telemetry exists and is trustworthy.
Should we use vendor default rules?
As a starting inventory to review, not as a deployed set. Enable what you have tuned and can action; leave the rest off.
How much ATT&CK coverage is enough?
Wrong question. Depth on the techniques relevant to your threat profile beats breadth across the matrix.
What breaks detections most often?
Silent log source failures and schema changes after platform upgrades. Monitor feed health as a first-class signal.
Key takeaways
- Treat detections as version-controlled, tested, owned artefacts.
- False positives are the failure mode, not a nuisance.
- Validate detections by triggering them deliberately.
- Technique coverage counts are a vanity metric; alert-to-incident ratio is not.