How IP Frog plans to report ISP performance by region
A preview of the sample thresholds, statistics, privacy rules, and caveats IP Frog will use before publishing regional ISP comparisons.
IP Frog is beginning to collect anonymous speed-test summaries that can eventually answer an interesting question: what performance are people actually observing from different networks in broad regions?
The word eventually matters. A colorful chart is easy to make. A comparison worth trusting needs enough data, clear definitions, and visible limitations.
What a report would group
The basic grouping key will be the autonomous system number, or ASN, attached to the request by Cloudflare. ASN is generally more consistent than the organization label, though one ISP can operate several ASNs and one ASN can cover multiple products.
Regional grouping may use country, first-level region, and—only when the sample is large enough—approximate city. IP-based cities are estimates, so broad regions will usually be more reliable.
Results will also be separated by meaningful factors we actually know, such as test mode and time window. We will not pretend to know a customer's plan tier, Wi-Fi setup, or access technology when the test does not supply that information.
Minimum sample sizes
IP Frog will not publish a provider/city row because two people happened to run a test. The initial target is at least 25 completed tests in a segment, with higher thresholds for rankings or strong claims.
The report should always show the sample count. A median based on 40 observations and one based on 40,000 observations do not deserve the same confidence, even if the displayed number happens to match.
Repeated tests can also overrepresent one connection. The privacy-first per-test identifier intentionally avoids a persistent person profile, so reports will need conservative outlier and repetition checks without attempting to identify individuals.
Medians before bragging rights
Simple averages are easily pulled around by a handful of extremely high or low values. Regional summaries should lead with the median and include a spread such as the 25th and 75th percentiles.
Useful report fields may include:
- Median download and upload speed
- Median latency and jitter
- Middle-range performance, not only one number
- Completion and failure rates
- Sample count and collection period
- Time-of-day comparisons where volume supports them
Networks that need special handling
VPNs, cloud providers, corporate gateways, universities, and mobile carrier gateways can make the apparent network or location different from the user's retail connection. IP Frog will classify or exclude obvious non-consumer networks when that distinction is defensible.
Provider names will be normalized for display while preserving ASN underneath. Any decision to combine multiple ASNs under one brand should be documented.
What the reports will not claim
These will be observed IP Frog speed-test results, not a scientific census of every subscriber. People choose when to test, often because something feels wrong. Devices and Wi-Fi differ. Fast connections may be more likely to hit device limits. Rural and low-traffic regions may take longer to reach a publishable sample.
Those biases do not make the data useless. They mean the report needs honest labels and should focus on patterns supported by enough observations.
When will the first report appear?
When the data earns it. The collection pipeline is now in place, but IP Frog will wait for meaningful coverage rather than publishing an empty leaderboard. Until then, this methodology is the promise the eventual report must keep.
For more context, read how ASN and ISP information works and exactly what the speed-test event contains.