If you have an interest in measuring and interacting with the world round you thru the Internet of Things (IoT), there are a selection of cheap microcontrollers and microcomputers you need to use. There are additionally many sensors obtainable that join to those units to measure many points of the bodily world.
These sensors interface with the microcontroller boards utilizing the I2C message bus, which packages that run on the boards can entry utilizing open supply libraries in MicroPython, Java, C#, and different in style programming languages. These units and libraries make it very simple to create subtle data-collection programs.
To show how simple and highly effective that is, I constructed a greenhouse monitoring system utilizing the next parts that I bought from SparkFun:
Adafruit has very comparable choices and connection programs.
Getting to know Prometheus
One of the primary issues you are able to do to start out interacting together with your world is to gather and analyze knowledge acquired by sensors. Open supply software program makes it simple to gather, analyze, show, and even take motion in your knowledge.
The Prometheus household of functions makes it simple to gather, retailer, and analyze knowledge as a time collection of particular person occasions. I’ll briefly introduce the related components of the Prometheus structure; if you need to study extra, there are lots of nice articles about Prometheus on Opensource.com, together with An introduction to monitoring with Prometheus and Achieve high-scale application monitoring with Prometheus.
The Prometheus suite consists of the next functions, which could be plugged collectively in varied methods.
Prometheus
The major Prometheus service is a strong time-series database that runs on a general-purpose pc, akin to a Linux machine, cloud service, or Raspberry Pi (the Raspberry Pi four is really useful). A Prometheus occasion could be configured to repeatedly “scrape” varied file- and network-connected exporter providers (e.g., HTTP, TCP, and so forth.) within the Prometheus exposition format. A single Prometheus service could be configured to scrape a number of targets, every with a novel job identify. A scrape goal publishes knowledge within the type of occasions with a user-defined identify, timestamp, worth, and non-compulsory set of key-value annotations. If an information supply publishes knowledge with no timestamp, the scrape’s precise time is routinely added to the occasion when it’s saved. It may also be configured to speak with a number of Alertmanager cases operating on the identical host or one other host on the identical community.
Once occasions are revealed in a Prometheus service, they are often queried utilizing the Prometheus Query Language. PromQL queries can be utilized to create tables and graphs of occasions. They may also be used to configure alerts, whereby a PromQL question situation’s reality causes the Prometheus service to set the configured alert’s firing state as true; this alert will stay within the firing state so long as the situation is true. Once the situation turns into false, the alert firing state is ready to false.
Multiple cases of an exporting service can publish the identical metrics however differentiated by annotations to establish the sensor. For instance, in case you have three greenhouse screens, every can publish its temperature, humidity, and different metrics, annotated with one thing like greenhouse=1, greenhouse=2, or greenhouse=three. Graphs, tables, and alerts could be configured to indicate all cases for a selected metric or simply the metrics with particular annotations.
All metrics saved in Prometheus are annotated with the job outlined for the scrape goal within the configuration. Every scrape goal configured in a Prometheus service has a Boolean metric known as up, which is ready to true every time the service efficiently scrapes the goal and false when it can not. This is a helpful metric to make use of in PromQL queries to outline alerts when a service goes down.
Alertmanager
The major Prometheus service doesn’t act on alerts—it simply holds the alerts’ state as firing or not firing at any specific second. The Alertmanager service works with a Prometheus service to arrange notifications when alerts outlined in Prometheus are firing. One or extra Alertmanager providers could be configured to run on general-purpose computer systems on the identical community because the Prometheus service.
Alertmanager notifications could be configured to speak with varied exterior programs, together with e-mail gateways, net service endpoints, chat providers, and in style ticketing programs. Each notification could be templated to make use of varied attributes in regards to the occasion, together with all of its annotations, to supply the notification message.
Node Exporter
Node Exporter is a quite simple daemon that runs on a general-purpose pc host as an online service and exports knowledge about that host by way of HTTP within the Prometheus exposition format. It is programmed to supply many various metrics about its host, akin to CPU and reminiscence utilization, utilizing logic outlined for every particular host structure (e.g., proc filesystem, Windows Registry, and so forth.).
A Node Exporter occasion may also be configured to current a number of Prometheus exposition format compliant recordsdata on the host filesystem. This makes it helpful for publishing metrics produced by one other software operating on the identical host. The instance greenhouse monitoring system makes use of a Python program to gather knowledge from the sensors and produce a Prometheus-formatted export file, and Node Exporter publishes these metrics.
Pushgateway
A Raspberry Pi Zero, three, or four can host a Node Exporter, however different microcontrollers (akin to an Arduino or Raspberry Pi Pico) can not. Pushgateway allows these units to publish their metrics. It is a microservice that may run on one other general-purpose pc host (akin to a desktop, a cloud, or perhaps a Rasberry Pi Zero, three, or four) and current a prometheus exposition formatted feed for a Prometheus service to scrape, and a REST API that different processes linked to its community can use to report customized metrics.
A Pushgateway occasion can run on the identical host because the Prometheus service or a unique host on the identical community. If the microprocessor can talk with the community utilizing the Pushgateway and Prometheus providers (e.g., an Ethernet cable, WiFi, or LoRaWAN), the method operating on the microcontroller can use a typical HTTP library to report metrics utilizing the Pushgateway REST API as a part of its course of loop.
Grafana
Grafana shouldn’t be a part of the Prometheus suite. It is an open supply observability system designed to drag in knowledge from a number of exterior knowledge sources and combine the info into customizable visualization dashboards. Grafana can pull knowledge in from quite a lot of exterior system sorts, together with Prometheus. It’s one other highly effective, open supply software that you need to use to create subtle dashboards with the info produced by your units. Grafana may also be put in onto a general-purpose pc, akin to a desktop or a Raspberry Pi Zero, three, or four. (I put in it on the Raspberry Pi four that hosts the Prometheus and Alertmanager providers.)
There are loads of tutorials obtainable that will help you rise up and operating with Grafana, together with a number of on Opensource.com, akin to The perfect combo with Prometheus and Grafana, and more industry trends and Monitoring Linux performance with Grafana.
Once Grafana is put in, use your browser to navigate to the Grafana host’s hostname or web protocol tackle (IP) at port 3000, and log in with the default credentials (clean / admin). Make certain to alter the admin password. You can then add an information supply and use the menu to decide on the Prometheus major server’s IP or host and port. Once you add the info supply, you can begin to graph knowledge from Prometheus or create dashboards.
If you’re putting in any of the above on a Raspberry Pi, make sure you obtain the Prometheus and Grafana binary distributions in your CPU’s structure. On a operating Raspberry Pi, you need to use both of those instructions:
uname -mcat /proc/cpuinfo
to get cpu structure. It will say one thing like armv7.
Connect the Raspberry Pi Zero’s sensors
Once you’ve gotten someplace to retailer the info, you possibly can assemble and configure the greenhouse monitoring system. I flashed the MicroSD card with the Raspberry Pi OS Lite picture and configured it for headless connection over WiFi. I plugged the Qwiiic pHAT onto the Pi Zero headers and linked the Qwiic cables from the Qwiic pHAT to every of the sunshine and environmental combo sensors. (Be certain to plug the yellow cable into the Qwiic pHAT on the facet with the Pi header connection and into the sensors on the facet with the I2C solder connection holes.) It can be doable to daisy-chain the sensors in case you have just one Qwiic connection to your Raspberry Pi.
Once the Raspberry Pi is linked to the sensors, plug the SD card into its slot, join the ability provide, and energy it up. It will boot up, after which you must be capable to connect with the Raspberry Pi utilizing:
ssh pi@raspbberrypi.nativeThe default password is raspberry, however change it to one thing safer utilizing the passwd command. You can even use ping in your desktop to get the host’s IP tackle and use it as a substitute of the raspberrypi.native tackle. (This is beneficial in case you have a number of Pis in your community.)
Install Node Exporter
Install the Node Exporter software in your Raspberry Pi Zero by downloading the binary distribution in your structure from the Prometheus web site. Once it’s put in, configure it as a systemd service in order that it routinely begins and stops with the Raspberry Pi.
Install Python sensor libraries
Raspberry Pi OS comes with Python three, nevertheless it doesn’t embody the libraries required to work together with the sensors. Fortunately, there are Python libraries obtainable.
Install SparkFun’s official Qwiic_Py library to entry the sensors on the Environmental Combo breakout. If you’re utilizing Raspberry Pi OS Lite, it’s important to set up pip (the Python bundle installer) for Python three:
sudo apt set up python3-pipThe mild sensor doesn’t but have an official SparkFun or Adafruit Python bundle, however you may get an open supply vml6030.py package from its GitHub repo and duplicate it to /dwelling/pi to make use of it in your monitoring software. It relies on the official SparkFun Arduino library.
Install the greenhouse monitor code
The greenhouse_monitor.py script on this venture’s GitHub repo makes use of the Python sensor libraries to append metrics for ambient_temperature, ambient_humidity, and ambient_light each 11 seconds to a file named /dwelling/pi/metrics.promenade within the format Prometheus expects:
#!/usr/bin/python3from veml6030 import VEML6030
import smbus2
import qwiic_bme280
import time
import sysdef instrument_metrics(mild,temp,humidity):
metrics_out = open('/dwelling/pi/metrics.promenade', 'w+')
print('# HELP ambient_temperature temperature in fahrenheit', flush=True, file=metrics_out)
print('# TYPE ambient_temperature gauge', flush=True, file=metrics_out)
print(f'ambient_temperature temp', flush=True, file=metrics_out)
print('# HELP ambient_light mild in lux', flush=True, file=metrics_out)
print('# TYPE ambient_light gauge', flush=True, file=metrics_out)
print(f'ambient_light mild', flush=True, file=metrics_out)
print('# HELP ambient_humidity humidity in %RH', flush=True, file=metrics_out)
print('# TYPE ambient_humidity gauge', flush=True, file=metrics_out)
print(f'ambient_humidity ', flush=True, file=metrics_out)
metrics_out.shut()print("Starting Greenhouse Monitor")
bus = smbus2.SMBus(1) # For Raspberry Pi
light_sensor = VEML6030(bus)
environment_sensor = qwiic_bme280.QwiicBme280()if environment_sensor.is_connected() == False:
print("The Environment Sensor isn't connected to the system. Please check your connection", file=sys.stderr)
exit(1)
environment_sensor.start()
whereas True:
mild = light_sensor.read_light()
temp = environment_sensor.temperature_fahrenheit
humidity = environment_sensor.humidity
instrument_metrics(mild, temp, humidity)
time.sleep(11)
This could be arrange as a systemd service, /and so forth/systemd/system/greenhouse_montor.service:
[Unit]
Description=Greenhouse Monitor
Documentation=https://github.com/prometheus/node_exporter
After=network-online.goal[Service]
User=pi
Restart=on-failureExecStart=/dwelling/pi/greenhouse_monitor.py
[Install]
WantedBy=multi-person.goal
A Node Exporter may also be configured as a systemd service to publish the metrics file produced by the greenhouse_montitor.py script at /and so forth/systemd/system/node_exporter.service:
[Unit]
Description=Node Exporter
Documentation=https://github.com/prometheus/node_exporter
After=network-online.goal[Service]
User=pi
Restart=on-failureExecStart=/usr/native/bin/node_exporter
--no-collector.arp
--no-collector.bcache
--no-collector.bonding
--no-collector.btrfs
--no-collector.cpu --no-collector.cpufreq --no-collector.edac --no-collector.entropy --no-collector.filefd --no-collector.hwmon --no-collector.ipvs
--no-collector.loadavg
--no-collector.mdadm
--no-collector.meminfo
--no-collector.netdev
--no-collector.netstat
--no-collector.nfs
--no-collector.nfsd
--no-collector.rapl
--no-collector.softnet
--no-collector.stat
--no-collector.time
--no-collector.timex
--no-collector.uname
--no-collector.vmstat
--no-collector.xfs
--no-collector.zfs
--no-collector.netclass
--no-collector.powersupplyclass
--no-collector.stress
--no-collector.diskstats
--no-collector.filesystem
--no-collector.conntrack
--no-collector.infiniband
--no-collector.schedstat
--no-collector.sockstat
--no-collector.thermal_zone
--no-collector.udp_queues
--collector.textfile.listing=/dwelling/pi[Install]
WantedBy=multi-person.goal
Note that you may depart off all of the --nocollector.* arguments, and node_exporter will export numerous metrics in regards to the Raspberry Pi host and the greenhouse_monitor knowledge.
Once the systemd service definitions are in place, you possibly can add and allow them utilizing systemctl, and they’ll begin as quickly as your Raspberry Pi boots up and has a community:
sudo systemctl allow greenhouse_monitor.py
sudo systemctl allow node_exporter
You can troubleshoot these providers utilizing:
sudo systemctl standing $servicenameThe Python script and systemd service definition recordsdata can be found within the project’s GitHub repo.
Restart the Raspberry Pi Zero and begin monitoring
When the Raspberry Pi begins, it would begin greenhouse_monitor.py and the node_exporter service. You can go to the node_exporter service utilizing the IP or hostname of the Raspberry Pi operating the greenhouse monitor at port 9100 (e.g., http://$ip:9100). Refresh each 11 seconds to see new entries.
Configure the Prometheus server scrape endpoint
Once your greenhouse monitor’s Node Exporter is exporting metrics, you possibly can configure the Prometheus service to scrape it. Add the next strains to the prometheus.yml configuration file throughout the scrape_configs part (substitute the IP within the targets with the IP of the system operating the greenhouse_monitoring service in your community):
- job_name: 'greenhouse_monitor'
# metrics_path defaults to '/metrics'
# scheme defaults to 'http'.
static_configs:
- targets: ['192.168.1.12:9100']
Prometheus will routinely load the configuration file each few seconds and begin scraping your greenhouse monitor. You can confirm that it has began scraping (and get its up/down standing) by visiting the Prometheus net person interface (UI) targets web page at http://$prometheus_host:9090/targets.
If it’s up (and inexperienced), you possibly can question metrics within the Prometheus net UI graphs web page http://$prometheus_host:9090/graph.
Once you’re getting knowledge in Prometheus, you possibly can go to the Grafana service at http://$graphana_host:3000. I created a dashboard known as Greenhouse with the panels for the three metrics exported by the greenhouse monitor. You can set Grafana to indicate knowledge within the panels utilizing the time controls. I used to be capable of get the values for a 24-hour interval from midnight to 11:59:59pm on the identical day utilizing the format from: YYYY-MM-DD 00:00:00 and To: YYYY-MM-DD 23:59:59.
Notice the time of day when the solar was shining by means of a window onto the system?
What must you measure subsequent?
You have a treasure-trove of knowledge at your fingertips to look at the bodily world. Next, you could possibly configure Alertmanager to ship notifications by means of varied communication applied sciences (e.g., webhooks, Slack, Gmail, PagerDuty, and so forth.) when alerts configured in Prometheus are firing.
Now that you understand how to measure your world, the query turns into: What do you need to measure?
