isitmyispDocs

How the verdict works

This page is for technicians. It explains what is measured and why the verdict never blames anyone without evidence.

What it measures

In 30-second windows, with one ping every 2 seconds:

  • Your router (or routers, if there's more than one).
  • The ISP's first hop: the first router outside your network.
  • Several internet destinations (1.1.1.1, 8.8.8.8 and 9.9.9.9 by default), over ICMP and TCP (a handshake to port 443, no data).
  • Your DNS servers, with real queries.
  • Your services, tunnels and the router over SNMP, if you set them up.

From that it identifies the path's segments: your network, the ISP's access network, its backbone and what lies beyond. It uses address types and autonomous system (AS) numbers, not hop positions.

Real loss or ICMP noise

Routers deprioritise pings addressed to themselves, so mtr shows "loss" on a middle hop while traffic through it is perfectly fine. isitmyisp only counts loss that persists to the destination, and requires ICMP and TCP to agree.

It doesn't cry wolf

  • Statistical significance. One packet lost out of 50 isn't "2 % loss". Loss must be significant by a 95 % Wilson interval, with at least 2 packets lost.
  • Its own baseline. Latency is judged against what's normal for each destination, not a fixed number: it's abnormal when it's twice the usual and at least 30 ms more. What's normal is learned from healthy windows only and kept across restarts, so a problem in progress never becomes "normal".
  • Hysteresis. Each window lasts 30 seconds. An incident opens after 2 bad windows in a row and closes after 2 good ones. If the ISP's first hop looks affected in one window and not in the next, it's still the same ISP outage, not two.
  • Intermittent drops. A connection that drops and comes back (bad, good, bad, good…) never makes 2 bad windows in a row, and it's exactly what the ISP needs to hear about. So 3 bad windows within the last 5 minutes also open an incident, marked intermittent on the dashboard, in alerts and in the report, with how many times it dropped ("dropped and came back 6 times"). It closes only after 3 minutes in a row without drops.

Whose fault it is

  1. Your network: loss or abnormal latency to your own router that also shows up on the internet. If only the router "loses" pings while the internet works through it, the router filters ICMP (common on business firewalls) and nobody is blamed.
  2. Your ISP: every destination affected. If its first hop is affected too, it's the access network; otherwise it's the backbone or its upstream.
  3. Others: only some destinations affected. It isn't your ISP.
  4. DNS: everything works except name resolution. It's your DNS server.

When a service you added fails while the internet is healthy, the verdict is "that service". When the office is on the backup link, it says so.

Loops and broadcast storms

A cable closing a circle between two switches (or both ends of one cable in the same switch) sends frames round and round. Everything drops on and off and, from outside, it looks like the ISP. While it measures, isitmyisp looks for four signs that need no extra permissions:

  • Duplicate replies (DUP) to its own ping to your router, once a second: a frame that went round the loop arrives twice. 2 or more in a window count.
  • Broadcast and multicast on your network card (/sys/class/net/<interface>/statistics/multicast): a storm is over 500 packets per second, or 10 times the usual and at least 100 per second. The usual rate is learned from calm windows.
  • Your router's MAC flapping back and forth in the ARP table (A → B → A). A single change doesn't count: you may have swapped the router.
  • Broadcast per interface on the router, if you connected it over SNMP (ifHCInBroadcastPkts and ifHCInMulticastPkts): over 1000 packets per second on any interface other than the WAN.

If a sign shows up and something is failing too (your router or the internet), the verdict is "There's a loop on your network", on your side, with the evidence and what to check: recent cables between switches and switches without STP; unplug them one by one until it stops. If there are signs but nothing fails, the terminal mentions it as a hint and the verdict doesn't change.

The idea comes from Bruno Torres, from a real case: an intermittent outage that looked like the ISP was a loop in the LAN.

Evidence for every incident

When an incident opens, an mtr capture towards the affected destination is taken and stored with it. If the path changes, it's rediscovered. Incident evidence is kept about 13 months, per-destination measurements 35 days.

What the evidence rules out

Every ISP incident lists what the evidence rules out (your LAN, ICMP noise, a single site, a full link, a loop on your network) and what it can't. Without router data over SNMP, a full link stays as "can't rule out". Better to say so than to over-claim.

Edit this page