Prometheus

Prometheus is a an open source monitoring and alerting toolkit. Originally built by a couple of Soundcloud engineers to modernize monitoring.The core of Prometheus is a data model with time series data identified by metric name and key/value pairs and stored in a time series database.

Prometheus also includes:

• a flexible query language to leverage this dimensionality
• no reliance on distributed storage; single server nodes are autonomous
• time series collection happens via a pull model over HTTP
• pushing time series is supported via an intermediary gateway
• targets are discovered via service discovery or static configuration
• multiple modes of graphing and dashboarding

Hackernets Setup

We are using a fairly simple setup with all prometheus components running in docker containers. Since Prometheus relies on pulling metrics via HTTP the data gathering and conversion to the correct metric format has to be done by components outside of Prometheus. We use currently use different projects to gather data, but writing your own applications is very easy considering the extensive amount of language support that the Prometheus project has to offer.

Our containers:

[root@prometheus ~]# docker ps
CONTAINER ID        IMAGE                COMMAND                  CREATED             STATUS              PORTS                    NAMES
7e22a6c7bbc4        vmware_exporter      "/opt/vmware_exporte…"   8 days ago          Up 8 days           0.0.0.0:9272->9272/tcp   vmware_exporter
67c5bfbaa81b        blackbox_exporter    "/bin/blackbox_expor…"   9 days ago          Up 8 days           0.0.0.0:9115->9115/tcp   blackbox_exporter
c77043cc77fe        grafana/grafana      "/run.sh"                9 days ago          Up 8 days           0.0.0.0:80->3000/tcp     grafana
463e5b3663c7        prom/prometheus      "/bin/prometheus --c…"   2 weeks ago         Up 8 days           0.0.0.0:9090->9090/tcp   prometheus
809f4ab916c9        prom/node-exporter   "/bin/node_exporter"     2 weeks ago         Up 8 days           0.0.0.0:9100->9100/tcp   node_exporter