Post by 
July 21, 2026 | 
Network monitoring fails most teams not because they lack tooling, but because they monitor the wrong things. Ping checks and interface utilization graphs catch blunt failures, but the incidents that cause real damage – degraded API response times, intermittent connection resets, CDN origin pull failures – live in subtler metrics: TCP retransmission rates climbing from 0.1% to 3%, DNS resolution times doubling for a single upstream resolver, or BGP path changes that shift traffic through a higher-latency transit provider. This guide covers the full scope of network monitoring dependency visibility. That means which metrics actually predict failures, how to map the services your infrastructure depends on, and where standard monitoring setups consistently leave blind spots.
Network Metrics That Matter
Most infrastructure teams start with interface utilization and call it network monitoring. That covers maybe 20% of the failure surface. The metrics that surface problems before users notice them are more specific.
Latency
Round-trip latency tells you how long a packet takes to travel between two points. For internal east-west traffic between application tiers, anything above 1-2ms on the same datacenter fabric is worth investigating. For cross-datacenter or cloud interconnect paths, you need a baseline first. Then alert on deviations from that baseline rather than absolute thresholds. A path that normally runs at 18ms spiking to 45ms indicates a routing change or congestion event worth examining, even if 45ms sounds acceptable in isolation.
For a deeper treatment of latency measurement and what the numbers mean operationally, see our guide on network monitoring: latency, loss, and dependency mapping.
Packet Loss
Packet loss above 0.1% on a production path starts affecting TCP throughput through retransmission behavior. Loss above 1% is a service-affecting condition for most TCP-based workloads. Loss at 5% or higher is a hard failure for real-time or streaming workloads. The problem is that many monitoring systems only report loss from ICMP probes, which network devices often deprioritize under load – giving you false negatives precisely when the network is stressed.
TCP Retransmissions
TCP retransmissions are a more honest signal than ICMP-based loss checks. When the kernel retransmits segments, it reflects actual application-layer impact. Track retransmission rate as a percentage of total segments. A baseline of 0.05-0.2% is normal on healthy production paths. Sustained retransmission rates above 1% indicate path quality problems that will appear as application latency within minutes.
On Linux, `ss -ti` exposes per-socket retransmission counters. At the aggregate level, `/proc/net/snmp` and the `netstat -s` output under `TCPRetransSegs` give you system-wide retransmission totals to track over time.
Throughput and Connection Counts
Throughput saturation is obvious when a link hits 100% utilization, but the performance degradation starts well before that point. Most network hardware shows increased latency and buffer growth at 70-80% sustained utilization. Connection count growth often predicts service exhaustion before throughput does: if your load balancer normally handles 8,000 concurrent connections and you see 14,000, something upstream is holding connections open longer than expected.
For how these network metrics fit alongside CPU, memory, and disk in a unified monitoring model, see CPU, memory, disk, and network monitoring.
Internal Network Monitoring vs External Dependency Monitoring
Internal network monitoring covers the infrastructure you control: switches, routers, load balancers, inter-pod networking in Kubernetes, east-west traffic between application services. External dependency monitoring covers everything outside your control perimeter: upstream DNS resolvers, API providers, CDN peers, cloud provider endpoints, and transit ISPs.
These two domains need different approaches. Internal monitoring uses agents, SNMP polling, flow data (NetFlow, sFlow, IPFIX), and synthetic probes between known endpoints. External dependency monitoring requires probing from multiple vantage points, tracking response codes and resolution times from the perspective of external clients, and maintaining awareness of what happens when a third-party service degrades.
The failure to separate these domains creates a common blind spot: teams build thorough internal monitoring, then get blindsided by a payment gateway timing out or an auth provider returning 503s because they never instrumented the external dependency path. Understanding how monitoring agents work on Linux, Windows, and network devices is a starting point, but agents alone won’t cover external paths without deliberate synthetic probing architecture.
Service Dependency Mapping: What It Is, Why It Matters, How to Build One
A service dependency map documents which services your infrastructure components depend on, and in what failure mode those dependencies matter. It is not an architecture diagram. It is a runtime dependency graph with failure impact annotations.
Why Dependency Maps Matter During Incidents
During an incident, the first question is always: what does this failure affect, and what is causing it? Without a dependency map, engineers spend the first 15-20 minutes of an incident reconstructing service relationships from memory or tribal knowledge. That delay compounds into SLA breaches. A dependency map answers those questions before the incident starts.
For CDN and SaaS environments, dependency maps also prevent over-alerting. When an upstream DNS resolver degrades, a team without a dependency map may receive 40 separate alerts – one per service endpoint that relies on that resolver. A team with a proper map receives one alert: upstream DNS resolver degraded, with a list of affected services attached.
How to Build a Service Dependency Map
Start with your most critical services and work outward. For each service, identify: what it calls directly, what it depends on for authentication and authorization, what DNS names it resolves, which certificate authorities it validates against, and which cloud or third-party APIs it integrates with.
Automate discovery where possible. Tools like eBPF-based network tracers, service mesh telemetry (Envoy, Linkerd), and distributed tracing (Jaeger, Tempo) surface actual runtime dependencies rather than intended ones. These often diverge. Services call things developers forgot about, or inherited from library dependencies.
Store the map as structured data, not a diagram. A diagram becomes stale immediately. A machine-readable dependency graph can be queried during incidents and kept current through CI/CD pipeline hooks.
For microservices-specific dependency modeling, see our guide on microservices monitoring architecture. For distributed system dependency challenges at scale, how to monitor distributed systems covers the architectural tradeoffs in detail.
DNS Monitoring: The Upstream Dependency Most Infrastructure Teams Underinvest In
DNS is the dependency graph of the internet, and it fails in ways that look like application problems. A service returning connection timeouts may be experiencing DNS resolution failures, not network or application issues. A CDN origin pull failing intermittently may be resolving to the wrong IP due to a stale or poisoned cache. These failures are invisible unless you instrument DNS specifically.
What DNS Monitoring Covers
Effective DNS monitoring tracks resolution time per resolver, NXDOMAIN rates, SERVFAIL responses, TTL compliance, and answer correctness. Answer correctness matters more than most teams realize: a resolver returning a syntactically valid but wrong IP address for a critical hostname causes application failures that look like network issues at the destination.
Monitor from multiple resolvers independently. Your application servers may use different resolvers than your monitoring system. If you only check DNS from your monitoring host, you miss resolver-specific failures affecting production systems.
Common DNS Failure Patterns
Resolver timeouts spike when upstream authoritative servers degrade. This often happens during DDoS events against the authoritative nameserver, or during DNS provider incidents that do not propagate to status pages for 20-30 minutes. TTL violations – where resolvers cache beyond the record TTL during failovers – cause stale answer problems that persist after you believe a DNS change has propagated. Negative caching of NXDOMAIN responses can block legitimate traffic for the full negative TTL period after a misconfiguration is corrected.
For infrastructure serving CDN traffic, DNS resolver geographic distribution also matters. A resolver returning correct answers in your primary region may return answers with suboptimal latency for edge locations if GeoDNS routing is misconfigured.
BGP and Upstream Path Monitoring for CDN and Infrastructure-Heavy Environments
BGP path changes are one of the most impactful and least monitored failure modes in CDN and multi-homed network environments. When a BGP path changes, traffic shifts to a different upstream transit provider. That new path may have higher latency, lower quality, or capacity constraints that cause congestion for your traffic class.
What to Monitor at the BGP Layer
Track prefix announcements and withdrawals for your own ASN and for upstream ASNs you depend on. Monitor AS path length changes for your announced prefixes: a path that grows from 3 hops to 7 hops usually means traffic is taking a longer, suboptimal route. Watch for route flapping, where a prefix is repeatedly withdrawn and re-announced, causing repeated path changes that affect traffic stability.
For CDN operators specifically, monitoring which upstream providers are announcing your prefixes and verifying that route origin authorization (ROA) records are valid protects against BGP hijack scenarios where another ASN announces your address space and attracts your traffic.
Tools and Data Sources
BGPmon and the RIPE NCC routing information service provide external visibility into how your prefixes appear globally. Internal visibility requires pulling BGP table data from your border routers via API or route collector, then alerting on state changes. Looking glass servers at your transit providers give you a point-in-time view of how they see your routes, useful during active troubleshooting but not sufficient for automated monitoring.
For CDN environments handling traffic across multiple PoPs, correlating BGP path changes with latency increases at specific edge locations is how you distinguish “the path changed” from “the destination is degraded.”
What Goes Wrong: Network Failure Modes That Standard Monitoring Misses
Standard monitoring catches interface down, high utilization, and ping failures. The failure modes that cause the longest incidents are the ones that do not trigger those checks.
Asymmetric Routing
Asymmetric routing occurs when outbound and inbound paths diverge. This passes all standard checks because packets arrive successfully in both directions, but stateful firewalls, NAT devices, and some load balancer configurations break under asymmetric paths. TCP sessions reset intermittently, and the failures look like application bugs rather than network topology issues. You detect asymmetric routing by running traceroutes in both directions and comparing AS paths, not by checking interface status.
MTU Mismatch and Path MTU Discovery Failures
MTU mismatches cause failures that appear only with large payloads. Small packets traverse the path successfully, so health checks and small API calls succeed while large file transfers or bulk data operations fail. Path MTU discovery relies on ICMP “fragmentation needed” messages, which many firewalls block. The result is a path that looks healthy under all standard checks but drops traffic silently above a certain packet size. This affects VPN tunnels, encapsulated traffic, and any path with mixed MTU configurations.
Partial Connectivity and Traffic Blackholing
A route can be present in a routing table but blackholed at a specific hop, where packets are accepted but never forwarded. This is common after BGP misconfigurations or during hardware failures on specific line cards. The route exists, the interface shows up, but traffic to specific destinations disappears. Standard SNMP polling and interface monitoring will not catch this. You need synthetic probes to specific destinations combined with traceroute validation to identify blackhole conditions.
Congestion on Non-Monitored Paths
Most teams monitor edge interfaces but not internal fabric links. Spine-leaf switches in a datacenter fabric carry aggregated traffic from multiple ToR switches. A congested spine link degrades traffic between multiple racks without triggering any per-interface threshold alerts on the edge links. Monitoring the fabric requires either flow telemetry from all fabric devices or synthetic probes between servers across different rack segments.
DNS-Induced Cascading Failures
When a DNS resolver slows significantly rather than failing completely, resolution timeouts propagate into application connection pools. Connection pool exhaustion then causes application-level failures that look like resource saturation – high thread counts, response queue depth growing – rather than network failures. Without DNS-specific monitoring, the root cause analysis leads engineers to the application layer first, adding 30-60 minutes to mean time to resolution.
Building a Network Baseline and Alerting on Deviations
Absolute thresholds are a starting point, not an endpoint. A packet loss threshold of 0.5% works for some paths and is far too permissive for others. Effective network alerting requires baselines: statistical models of what normal looks like for each path and each metric, then alerts when observed values deviate significantly from that baseline.
How to Establish Baselines
Collect at least 2-4 weeks of metric history before setting deviation-based thresholds. Network behavior varies by time of day and day of week. A Monday morning traffic spike is normal; the same spike on a Sunday at 3am is anomalous. Your baseline model needs to account for these cyclical patterns, which means simple average-based thresholds will generate false positives during predictable traffic patterns.
For most network metrics, a useful alerting approach is to alert when a metric exceeds the mean plus two standard deviations for that time period, sustained for 5-10 minutes. Point anomalies that resolve in under 2 minutes rarely require human response. Sustained deviations nearly always do.
Tooling for Network Monitoring Baselines
Prometheus with recording rules handles most network baseline computation at reasonable scale. Recording rules pre-aggregate metrics into time-of-week-aware summaries, which you can then reference in alert expressions. For larger environments, purpose-built anomaly detection in tools like Grafana Mimir or Elastic supports more sophisticated statistical models without building them by hand.
The same baselining discipline applies to other system resources. Our guides on disk I/O monitoring and memory monitoring cover analogous baseline approaches for those subsystems. For a full Prometheus-based implementation including Alertmanager configuration for network alerts, see the Linux monitoring stack with Prometheus, Node Exporter, Alertmanager, and Grafana guide.
How BlueGrid.io Does the Network Monitoring for CDN and Infrastructure Clients
BlueGrid.io’s NOC processes over 50 million monitoring events per month across CDN providers, SaaS platforms, and infrastructure-heavy environments. Network dependency visibility is one of the more operationally complex areas we cover because it requires correlating signals across layers: BGP path data, DNS resolution metrics, synthetic probes, flow telemetry, and application-layer indicators simultaneously.
For CDN clients specifically, we maintain synthetic probes to origin infrastructure from multiple external vantage points. These catch scenarios where the CDN edge is healthy but origin pull paths degrade – a failure mode that passes all internal checks but appears as cache miss latency to end users. We track DNS resolution times for all critical hostnames from the resolver sets that production infrastructure actually uses, not just from the monitoring host.
BGP path monitoring runs continuously for client ASNs, with alerting on prefix withdrawals, AS path length increases, and route origin mismatches. When a BGP event correlates temporally with application latency increases, the NOC team has enough data to confirm causation rather than guess at it. That distinction matters for routing decisions: withdrawing a peer session or adjusting local preference based on confirmed data is very different from making those changes speculatively during an active incident.
Our 24/7 monitoring model operates with a 1-hour incident response SLA. For network dependency events, that SLA covers not just alert acknowledgment but initial root cause hypothesis and escalation path. The network engineering team supports complex BGP and routing investigations that go beyond NOC-level triage.
If your team needs broader coverage of managed infrastructure monitoring across network, compute, and storage layers, BlueGrid.io’s service catalog covers the full stack.
