Cool! It's so funny my internet has been bad just recently so I took an old bash script of mine which did something similar and have been upgrading it into a proper application which exports metrics to Prometheus and Grafana: https://github.com/Noah-Huppert/net-test
I saw a comment below where some was rolling their eyes that you "complicated" stuff with Prometheus, Grafana, and Docker and how you could just use Bash scripts and crons. As I just upgraded my codebase from this more bare metal approach to this "more complex setup" I'd like to mention: there's no way you could do time series statistical analysis easily with "just a cron job and a bash script". Prometheus and Grafana are for more than just buzz words. Prometheus offers an advanced time series database which allows you to, at minimum, do more robust analysis using data techniques like Histograms. As for Grafana, it makes exploring data dead easy.
Providing users with a Docker Compose setup is also something I did with my tool and the benefits are huge. It lets me distribute a setup which relies on multiple moving parts working smoothly together. Sure I could write a whole wiki on how you should setup Prometheus Grafana and my tool, or I could distribute the setup with a configuration as code tool. Ensuring that even if someone doesn't want to use Docker Compose they can at least read my configuration as code and see exactly what I did to setup my tool.
Recently I was looking to add some metrics to a small script and I didn't want to deal with all the complexity of setting up grafana, prometheus, the push-proxy-thing (sorry, don't remember the name).
Looking for alternatives, I found InfluxDB, which was just one apt-get install away.
It comes with a web interface to create dashboards, supports push based metrics, and overall I'm loving it.
The web interface doesn't seem as powerful as grafana, but it covers most basic needs. And I think you can use it with grafana too.
You can indeed; I use InfluxDB and Grafana to provide some stats on my home server. Pretty easy to set up, and Grafana just uses the Flux query language to pull the data from Influx.
I've been tangentially looking at this as a 'beta' customer of Starlink, as the service is presently extremely variable with many small outages throughout the day. This kind of a collection of internet monitoring is super useful for the individual, but what I would really love to see is a way to federate the data in a meaningful way so that you can start to see operating patterns in the provider(s) being used.
We do something similar at https://imup.io but we run ping & speed tests from your home PC rather than shipping you a device. We approach the situation wanting to save folks money on their internet services for degradation in addition to providing analytics.
Thanks for sharing that link for Ripe though, super cool project.
Use node exporter on your machines and Prometheus to scrape the metrics and you will be drowning in pretty graphs. Super easy to set up and gives you system details across all your machines, all in one place.
There are also script exporters (a few implementations) for simpler Nagios style checks and more specialised ones for getting things like Cloudwatch data (shout out to YACE[1] - with dashboards from promcat). It's really quite vibrant the ecosystem.
Of course but it's still in the traditional domain of networking kit (generally). As there are many components virtualised or software defined then I think in those areas, not so much as there are alternative ways to get the metrics. SNMP is a bit long in the tooth and can be a bit unweildy sometimes, ymmv etc.
I recently set up smokeping to gather some data, I can't say it was a breeze - documentation appears complete but misses some key pieces of information you have to dig for or trial/error it out.
E.g: i struggled figuring out where to put a non-standard probe
Failure modes were also quite weird... adding a slave to the mix broke my mind a little. Was never sure it worked but then didn't need it so retired it before i figured it out
Changing what you think is something trivial (e.g: frequency of data collection) invalidates your previously collected data files.
Other than that - fairly solid, did what i needed, runs without a fuss.
For simpler needs, like being able to see when my network dropped out, without needing another device... On a OpenWrt home router, I usually install `collectd-mod-ping` and `luci-app-statistics`, and have it periodically ping a couple of different hosts that will answer pings reliably. (This can double for monitoring your third-party hosting.)
Caveat: since you're probably running OpenWrt on a consumer SoHo router, I recommend sending this logging data to a cheap USB flash drive, not wearing out the router's onboard flash.
An even simpler alternative would be to keep a browser window open with the OpenWrt admin UI's "Status -> Realtime Graphs" displaying.
(If you're new to OpenWrt, a fairly recent router hardware model that's easy to re-flash with solid OpenWrt, and fairly inexpensive: https://openwrt.org/toh/netgear/r7800 )
Interesting lots of people had the same idea. I have done something similar for my home network using an Ethernet connected RaspberryPI dedicated to this task using a cronjob, the speedtest-cli JSON format, MongoDB and just some JS graphs. - very rudimentary.
One thing I found very important was to plot the scatter of PING vs. Download speed (my Upload is super stable).
Then I created the probability density function of that (using KDE) and used it to get a better deal from my network provider - I could prove they were selling me a slower network most of the time - they now allocate 70Mb/s on a 50Mb/s deal so just that they hit the 50Mb/s most of the time.
It also allowed me to clearly identify some outlier situations where my network suddenly became 'bad' and I could show evidence of it being bad over time.
Certainly a project that should see more traction!
I've been toying around with the idea of building a small device or widget to display things like this. There is hardware like this for temperature and humidity and such but nothing for internet health.
If anyone is interested feel free to contact me. Would love to learn about what you would want in such a device.
Sounds similar to the SamKnows Whitebox, which is a MT7621+MT7603+MT7612 router from which they run their speedtests and monitor various metrics. They don't actually run clients off those radios, they just scan and sniff.
I don't think regular consumer is their target audience however, they seem more geared to ISP's and government agencies who need performance data over a large amount of connections.
I suppose the problem with wanting to build a small device that does this, is finding the balance between cost-effective and actually being able to pull off 1gbit throughput on it.
Of course, this all assumes you were going the repurposed hardware route as opposed to design from scratch. Doing it from scratch would have it's own costs and pitfalls. (Board design, manufacture, licensing costs, etc)
Cool. I have read a lot of approaches to home network monitoring and everybody seems to do things a little bit different.
Telegraf has built-in modules for ping and dns latency. I recommend it as an alternative to prometheus.
Telegraf is just going to do the "probe" component, right? You're still going to need Prometheus or Influxdb for storing the timeseries, and Grafana for visualizing I expect.
Grafana and prometheus are a very nice combo for all sorts of monitoring. For my current setup I have added victoriametrics which is a drop-in replacement for prometheus. I like it more for home setup as it is easier on disk-space usage and the collection client (vmagent) does disk buffering. So I run the collection on a battery backupped raspberry pi which uses remote_write to send it to victoriametrics on my home server (which has a less higher uptime than the raspberry pi) without the risk of losing precious metrics.
I use telegraf with Ping and speedtest cli that both output to influxdb which then I browse with Grafana as well. Kind of similar but not quite. I also orchestrate with docker-compose on my RPi and I find this quite pleasent but in the end you should choose what you’re most familiar with. I didn’t want to spend my innovation tokens on promotheus hence I’ve gonna in what I viewed them as simpler, with telegraf.
I ping Google, CloudFlare, Quad9 and Amazon EC2 and plot it to see the any unusual spikes.
You can still do it, but the "old school" way of doing it is much less flexible.
For this kind of setup, prometheus and grafana are not complicated (close to 0 configuration), and docker mostly works out of the box on linux hosts, barring nftables shenanigans. You end up with something that "just works", is pretty, and does not require fiddling.
Technically I do find this overkill for "home network monitoring", but at least it does not require k8s so I won’t complain.
One would think that should be a hell of a low bar. It’s like having to choose a car and going “Trabant will do, at least it does not require a crew of seamen”.
As someone who has set up a number of "simple" visualizing systems using the exact toolpath you mention (cron, shell scripts, rrdtool), I can say with some authority that getting them right was never simple.
And the load of dozens to hundreds of data points in rrd is shockingly large. Our infrastructure utilization dropped by 30% when I switched from a Graphite/collectd setup to gathering MORE data more frequently with telegraf and influxdb.
rrd and cacti are considerably more complicated and far less useful than prometheus and grafana.
I generally agree with the rest of your point, but the new-fangled infrastructure tools do have some merit and I look forward to seeing where we go with them.
Don't be fooled by the rain of downvotes that some comments are getting: the HN crowd is incredibly biased towards hyped stuff and CV-driven development.
I worked in FAANGS where simplicity really mattered over hype.
Prometheus as a monitoring tool is definitely a lot more complex and does not 'just work' straight out of the box. You need to configure Prometheus to know where to scrape using service discovery, set up different agents on your hosts to export metrics, build graphs in Grafana using PromQL to look for specific labels, and create alerts Alert Manager for every alert you want to receive.
You don't even need cacti or rrd if it's just taking a value and putting it in a file. However these newer tools are much more flexible that cacti and RRD. I find using Grafana miles better than dealing with RRDs, beter granularity and more options for querying data/overlaying results from different metrics on the fly etc.
A lot of people are now building huge container based SaaS software in their $Day_job, so in their free-time they use those technologies because they are more familiar, or they want the practice.
I ran K3s for a few weeks on ODroid hosts at home to get more k8s experience.
I would use Telegraf (a better/distributed collector far more efficient than cacti, even if you have to learn the OIDs for the snmp devices you want to monitor, because not everything is snmp anyway), InfluxDB (a timeseries database better than rrd, is not the only alternative, but at least it works for me), and yes, Grafana.
The cron gives you low granularity (and for some things, it may matter to measure things that changes in less than a minute), and flexibility (both in what you can measure and how you can arrange and correlate the different pieces of information).
Well, FWIW, I’m running a simple smokeping Docker container.
Just works, sits neatly alongside a few extra services (managed via compose) in a little ARM dev board and has kept on ticking for around five years without any hassles using exactly those bits you mentioned.
The new things are shinier, I think, and on the other hand a lot of the old, simpler tools have become less relevant.
As an aside, I recall a younger self annoyed at the inability to get data out of RRDTool into a nice SVG chart. These days I just want the data in whatever format the best tool spits out...
See my other comment in this thread. I've found InfluxDB a lot easier to setup and use (just apt-get install) and comes with push support, dashboards web ui, etc.
I'd say it is easier and more powerful than rrd and cacti (used them like 10 years ago!)
They called to stay popular because they kinda suck. The setup you described works on one host and produces static graphs.
With Prometheus and Grafana you can monitor an entire data center and add visualizations for metrics as you need them with no change to the monitored hosts.
TLDR: the old software stack could be made to do this, modern monitoring stacks do it out of the box.
I think they meant to give an example to indicate the flexibility of the platform. You know nobody is claiming people regularly run data centers at home — why are you asking?
I saw a comment below where some was rolling their eyes that you "complicated" stuff with Prometheus, Grafana, and Docker and how you could just use Bash scripts and crons. As I just upgraded my codebase from this more bare metal approach to this "more complex setup" I'd like to mention: there's no way you could do time series statistical analysis easily with "just a cron job and a bash script". Prometheus and Grafana are for more than just buzz words. Prometheus offers an advanced time series database which allows you to, at minimum, do more robust analysis using data techniques like Histograms. As for Grafana, it makes exploring data dead easy.
Providing users with a Docker Compose setup is also something I did with my tool and the benefits are huge. It lets me distribute a setup which relies on multiple moving parts working smoothly together. Sure I could write a whole wiki on how you should setup Prometheus Grafana and my tool, or I could distribute the setup with a configuration as code tool. Ensuring that even if someone doesn't want to use Docker Compose they can at least read my configuration as code and see exactly what I did to setup my tool.