Statistics

Most applications generates some sort of statistics, either as logs or system metrics. The statistics can be used to gain valuable information on how to improve the application. For a homelab, this piece of information is useful to see the overall usage of the homelab, especially if something specific is happening regarding the resources used, but also to keep track on who is using the services provided. The type of statistics I am using is mostly existing softwares, but I have made some small additional changes due to the design of the infrastructure, some which has been mentioned previously.

Prometheus, and its driver Grafana, are two common metrics holder and visualizer. An exporter is required, default for any system a node-exporter could be used ¹. For specific services and custom data, there exist plenty of libraries out there ². Prometheus will either act as a host and listen on exporters, or most commonly have a list of hosts to check in a stated interval. It will store as much metrics specified, but sadly this is not entirely true, and it will try to purge older data whether one like it or not. Grafana on the other hand is just a visualizer of the data, connecting to Prometheus running some query and display the results as fancy graphs, numbers and even alarms. This combination is very good whenever a handy system like PVE does not exist on all hosts, or even in a cluster.

GoAccess is an odd one out, but it is truly awesome if you run any type of standard web service. It basically goes through the access logs for all or one domain, and generates an HTML file to be displayed. It either do this statically once, or iteratively follow the logs and updates the site on a public endpoint. It can also do this in the terminal as well, for the sysadmins who live in the terminal. I myself picked to generate static HTML pages individually for each (sub)domain. It will display all requests, failed and static ones, HTTP responses, visitors, operating systems, browsers, and time distribution.

PVE may only be seen as a layer on top of KVM ³ and LXC ⁴, in addition to several other useful tools. However, some of these tools are created in-house, giving not only the useful WebUI, but also a couple of statistics metrics, like graphs and values. These are essential for live-usage, and while you can occasionally see these for up to a year back, it is mostly used for at most a week to gain some insight in the daily usage of the system. It is a great tool to get an overall overview of your services on that specific node or a whole cluster.

VyOS, while mainly being just a router, also got some great statistics on a couple of its services. This includes bandwidth throughput, firewall hits, DHCP leases, NAT packages and a lot more which I currently am not using.

With all this information I can get a good overview of what is used in my infrastructure, either per application, or the overall usage for the whole network. Learn to use them, and make use of the information in order to improve what you have further.