AIOps and Anomaly Detection
Statistical and machine-learning approaches to alert correlation, anomaly detection, and noise reduction — with honest limits on what they deliver.
AIOps is a vendor category more than a technique, which makes it hard to evaluate. Underneath the label are several distinct capabilities with very different maturity levels, and it is worth separating them before buying or building anything.
What is actually in the box
| Capability | Maturity | Realistic value |
|---|---|---|
| Alert correlation and grouping | High | Substantial — collapses storms into incidents |
| Anomaly detection on metrics | Medium | Useful for seasonal metrics where thresholds fail |
| Log pattern clustering | High | Substantial — turns log volume into a short list |
| Change correlation | High | Substantial, and mostly not machine learning at all |
| Root cause identification | Low | Suggests candidates; does not identify causes |
| Predictive failure detection | Low to medium | Works for degradation with a physical trend (disks, capacity) |
| Automated remediation selection | Low | Use deterministic rules instead |
The high-maturity items are where the return is. Correlation and clustering are well-understood problems with reliable techniques, and they attack the biggest practical problem in operations: too many signals.
Alert correlation
During a real incident, one root cause generates dozens or hundreds of alerts. Correlation groups them into a single incident so responders see one thing.
flowchart TD
subgraph "Raw alerts"
A1[db-primary unreachable]
A2[payments-api 5xx]
A3[orders-api 5xx]
A4[checkout latency P99]
A5[queue depth rising]
A6[cart-service timeouts]
end
A1 & A2 & A3 & A4 & A5 & A6 --> C[Correlation engine]
C --> D["One incident:<br/>database primary failure<br/><br/>6 alerts · 5 services affected<br/>most likely origin: db-primary"]
The correlation signals that carry the most weight, in order:
- Temporal proximity — alerts within a short window are probably related.
- Topological relationship — the service dependency graph tells you that payments depends on the database, so a database alert plausibly explains a payments alert. This is the strongest signal, and it comes from your service catalogue rather than from a model.
- Historical co-occurrence — alerts that have fired together before.
- Textual similarity — same error class or message pattern.
The topology signal is worth emphasising because it is the one that requires no machine learning and delivers most of the value. If you have an accurate dependency graph, rule-based correlation gets you a long way. If you do not, no amount of statistical sophistication compensates — which makes the service catalogue the prerequisite investment.
Anomaly detection on metrics
Static thresholds fail on metrics with strong daily and weekly seasonality. A request rate of 1,000/s is normal at 2pm on a weekday and alarming at 3am on a Sunday.
flowchart LR
A[Historical metric series] --> B[Decompose:<br/>trend + seasonality + residual]
B --> C[Predict the expected range<br/>for the current period]
C --> D{Observed value<br/>outside the band?}
D -->|Yes, sustained| E[Anomaly]
D -->|No| F[Normal]
E --> G{Does it map to<br/>user impact?}
G -->|Yes| H[Alert]
G -->|No| I[Record only]
Where this earns its place:
- Business metrics with strong seasonality — orders per minute, sign-ins per hour.
- Metrics where “normal” drifts as the product grows, making static thresholds need constant retuning.
- Detecting the absence of expected activity, which threshold alerts handle badly. A batch job that did not run produces no metric at all, and a “value is too low” threshold is awkward to set.
Where it disappoints:
- On core service health. SLO burn-rate alerting is better: it is explainable, tied directly to user impact, and does not need training data. Use anomaly detection to supplement SLO alerting, not to replace it.
- On low-volume or highly irregular metrics, where there is no stable pattern to learn.
- After a legitimate step change. A successful marketing campaign or a new large customer produces a genuine anomaly, and the model will flag it daily until it re-learns.
The rule that keeps this useful: anomalies inform; SLO breaches page. An anomaly is “this looks unusual”, which is not the same claim as “users are being harmed”.
Log pattern clustering
Millions of log lines usually contain a few hundred distinct patterns. Clustering extracts templates by identifying the variable parts:
Raw:
"Connection to 10.0.4.17:5432 failed after 5001ms (attempt 3)"
"Connection to 10.0.4.19:5432 failed after 5002ms (attempt 1)"
"Connection to 10.0.4.21:5432 failed after 4998ms (attempt 2)"
Template:
"Connection to <ip>:<port> failed after <duration>ms (attempt <n>)"
→ 14,203 occurrences in 5 minutes · NEW pattern, first seen 09:14Z
The genuinely valuable output is not the clustering itself but new pattern detection: a log template that has never appeared before, appearing right after a deployment, is one of the highest-signal indicators available. It catches errors nobody wrote an alert for.
Change correlation
The most useful capability in the whole category, and the least sophisticated: when an anomaly or incident occurs, automatically list every change in the affected blast radius in the preceding window.
Deployments, configuration changes, feature flag flips, infrastructure applies, dependency updates, and certificate rotations — correlated by service dependency and time. Since change is the leading cause of incidents, this answers the first question of nearly every investigation. It requires no machine learning; it requires a change log and a dependency graph.
Being honest about root cause
Vendors sell “root cause analysis”. What these systems produce is a ranked list of correlated candidates, which is genuinely useful and is not the same thing. Correlation cannot establish causation, and in a distributed system the true cause is often several hops from the symptom and sometimes outside the telemetry entirely.
Present it accordingly: “these three changes and two upstream alerts are correlated with this incident” is honest and helpful. “Root cause: service X” is neither, and it sends responders down the wrong path with false confidence.
Prerequisites
Every capability here depends on data quality, and this is where most AIOps implementations actually fail:
- Consistent metadata. Service names, environments, and owners must match across metrics, logs, traces, alerts, and the change log — otherwise nothing correlates.
- An accurate dependency graph. The single highest-value input, and the one most often stale.
- A complete change log covering deployments, config, flags, and infrastructure.
- Sufficient history. Anomaly detection needs weeks of clean data; a metric introduced last Tuesday has nothing to learn from.
Investing in these fundamentals typically returns more than any algorithm on top of them. A correlation engine fed by an inaccurate dependency graph produces confident nonsense.
Adoption checklist
- Service metadata is consistent across all telemetry and the change log.
- An accurate, maintained dependency graph exists.
- Alert correlation groups storms into incidents using topology first.
- SLO burn-rate alerting pages; anomaly detection informs.
- New log pattern detection is enabled and routed to owners.
- Change correlation is attached automatically to every incident.
- Correlated candidates are presented as candidates, never as root cause.
Last updated 19 Aug 2026, 00:00 UTC.