Recursive DNS+AD-Blocker — Part 4: pihole2influxdb2 — How to monitor your Pi-hole servers
In this article I will show you how to collect and send Pi-hole stats to an influxdb2 bucket by using a simple Python script pihole-to-influxdb2.py or my giannicostanzi/pihole2influxdb2 docker image you can find on DockerHub
As I’ve shown at the end of my Recursive DNS Resolver with AD-Blocking Features — Part 1 article, I’ve implemented some Grafana Dashboards to monitor the performance of the Pi-holes services running in my home network, one on a Raspberry Pi3 and one on a Synology NAS. Let’s see how you can collect stats via Pi-hole web API.
Pihole Web API
Pi-hole service exposes a GUI that you can use to configure it and to view some performance data on standard HTTP port 80/tcp. The same port is used also to expose information in a JSON format, just visit http://your_pihole_server:port/admin/api.php
Obviously you can omit :port if you did not change the default port for HTTP GUI.
Here you can see the data formatted via python json.dumps(data, indent=4):
Python API interaction and InfluxDB2 upload
JSON format can be very easily used with Python, so I’ve build a script that collects data from the given Pi-hole servers and then uploads to my InfluxDB2 container running on Synology NAS. You can find the whole script on my GitHub.
Script Environment Variables
The script relies on some environment variables:
INFLUX_HOST="my_influx_ip_or_dns_name"
INFLUX_PORT="my_influx_port"
INFLUX_ORGANIZATION="influx_organization_name"
INFLUX_BUCKET="influx_bucket"
INFLUX_TOKEN="influx_token"
INFLUX_SERVICE_TAG="pihole"
PIHOLE_HOSTS="x.x.x.x:80:raspberry,y.y.y.y:50080:nas"
RUN_EVERY_SECONDS=60
VERBOSE="false"Influx host and pihole hosts can be IPs or names resolved via DNS.
Influx organization, bucket and token are all parameters set on your InfluxDB2 instance.
Influx service tag is, as the name suggests, the value set to the tag service within your measurements, along with a host tag that will contain raspberry and nas in my example above, which are the names I’ve specified in PIHOLE_HOSTS environment variable.
I’ve used environment variables because this script will be embedded in a Docker container as I will explain in a following section.
I’ve prepared a shell launcher that exports the environment variables, otherwise you can directly set them within the python script:
If you pass the -t flag to the launcher script, it will pass it to pihole-to-influxdb2.py.
You can find it also on my GitHub:
Script execution
The script executes forever querying the Pi-hole hosts in PIHOLE_HOSTS every RUN_EVERY_SECONDS seconds.
You can run the script with -t flag in order to retrieve data from Pi-hole and print it in a well-formatted easy-to-read form, without uploading it to InfluxDB2.
The script connects to the Pi-hole web API and retrieves the JSON data previously shown:
We can not directly upload the stats python dictionary obtained with the above command, because there is a nested gravity dictionary with Python gravity db stats, so I pop the gravity_last_updated dictionary out of the stats dictionary and build my own gravity stats from the retrieved dictionary:
Then I upload the stats and gravity measurements to InfluxDB2 with the write api provided by influxdb_client python module (install it with pip install influxdb_client):
Note: if you give a look at the whole script on GitHub (which is always up-to-date, respect to the above code snippets which may become obsolete) there is a forced conversion to float for ads_percentage_today value because it is almost always a float but it can sometimes be zero (if the Pi-hole server has not blocked anything today) and Influx Write API would complain if we try to upload a zero (which is uploaded to the bucket as an Integer) instead of a float, because the field that it already finds on the target bucket is almost always initialized as a float.
The script creates a healtcheck file in the script directory which is used for container healthcheck status management. It is set to FAILED if there is a problem communicating with the InfluxDB2 server or if one of the Pi-hole servers can’t be reached, otherwise it is set to true.
Exploring data on InfluxDB2
Now, as long as the script is running, data will be uploaded to the InfluxDB2 bucket. You will find the following measurements and fields on your bucket:
As you can see we have gravity and stats measurements with their corresponding fields (file_exists and seconds_since_last_update are gravity fields, all the other one are stats fields). Each record has also a host and service tag, as explained before.
Visualizing data in Grafana
Now you can easily use the collected data to build your own Grafana Dashboard:
Just to give an example, you can visualize the cached VS forwarded info with the following Flux query:
I won’t go deep into more detail about InfluxDB2 and Grafana because it would require a whole new article, I just wanted to give an idea about what you can do with the collected data and let you experiment by yourself.
Let’s dockerize pihole2influxdb2
I’ve started using docker few weeks ago and since it is supported by my Synology NAS which is running 24x7x367, I’ve decided to build my own docker image with pihole-to-influxdb2.py script.
You can download it with the following command:
docker pull giannicostanzi/pihole2influxdb2:latestYou can find info about the container on pihole2influxdb2 Docker Hub page, where environment variables and usage info is detailed.
If you want you can execute a test run with the following command, which should give the same output as pihole2influxdb2.sh -t:
docker run -t --rm \
-e INFLUX_HOST="influxdb_server_ip" \
-e INFLUX_PORT="8086" \
-e INFLUX_ORGANIZATION="org-name" \
-e INFLUX_BUCKET="bucket-name" \
-e INFLUX_TOKEN="influx_token" \
-e INFLUX_SERVICE_TAG="pihole-test" \
-e VERBOSE="true" \
-e PIHOLE_HOSTS="ip1:port1:tag_name1,ip2:port2:tag_name2" \
pihole2influxdb2 -tIf the Influx host or a Pi-hole host runs on the same docker instance you can specify the respective container names, but this requires the pihole2influxdb2 container to be attached to the same non-default bridge network(s) of influxdb and pihole in order to have container-to-ip name resolution work as expected. For example I have this container running on my Synology NAS, attached to npm network (influxdb1 container) and services network (pihole-nocache container, my modified container which removes caching from pihole official one.. you can find it here and you can pull it with docker pull giannicostanzi/pihole-nocache):
You can create the container from the GUI or you can use the CLI (here I use container names influxdb1 and pihole_nocache as an example) to create the pihole2influxdb2-stats container from previously pulled image pihole2influxdb2:
docker run -d --name="pihole2influxdb2-stats" \
-e INFLUX_HOST="influxdb1" \
-e INFLUX_PORT="8086" \
-e INFLUX_ORGANIZATION="org-name" \
-e INFLUX_BUCKET="bucket-name" \
-e INFLUX_TOKEN="XXXXXXXXXX_INFLUX_TOKEN_XXXXXXXXXX" \
-e PIHOLE_HOSTS="rpi_IP_ADDR:50080:p-rpi,pihole_nocache:80:p-nas" \
-e RUN_EVERY_SECONDS="60" \
-e INFLUX_SERVICE_TAG="my_service_tag" \
giannicostanzi/pihole2influxdb2You can see debug info (enabled with the VERBOSE environment variable set to true, but it is not so verbose so you can enable it safely) and error messages from Synology Docker GUI by clicking on the running container and then on the logs tab or via CLI. For example I’ve shutdown my pihole-nocache container to see the error message printed in the logs and causing the pihole2influxdb2-stats container to become unhealthy:
docker logs pihole2influxdb2-stats -f
<urlopen error [Errno -5] No address associated with hostname>
Could not connect to pihole-nocache:80(nas)docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
22ff98ab4475 giannicostanzi/pihole2influxdb2:latest "python pihole-to-in…" About a minute ago Up About a minute (unhealthy) pihole2influxdb2-stats
pihole-to-influxdb2.py
You can find the script sources and Dockerfile for pihole2influxdb2 Docker Image on GitHub:
Conclusions
I hope you have found this article interesting, I enjoyed building my own container from a previously developed script that was scheduled via cron on my Raspberry. If you have question, just add a comment and I’ll try to help you and if you want to monitor also your Unbound servers, let’s move to the next arcticle Recursive DNS+AD-Blocker — Part 5: unbound2influxdb2 — How to monitor your Unbound servers! :)
