prometheus监控服务端口状态

实现思路：

借助blackbox exporter利用端口探测是方式实现监控服务端口。

实现步骤

下载地址：
https://github.com/prometheus/blackbox_exporter
下载tar.gz的安装包，这里演示0.16.0的版本
安装步骤：
1、解压安装包并赋权
tar -zxvf blackbox_exporter-0.16.0.linux-amd64.tar.gz -C /usr/local/
mv blackbox_exporter-0.16.0.linux-amd64 blackbox_exporter
mv blackbox_exporter /usr/sbin/
mkdir -p /etc/blackbox
cd /usr/local/blackbox_exporter
cp blackbox.yml /etc/blackbox/
useradd -g prometheus -s /sbin/nologin prometheus
chown prometheus:prometheus /etc/blackbox/ -R
2、配置自启动脚本：
vim /usr/lib/systemd/system/blackbox_exporter.service

[Unit]
Description=blackbox_exporter
After=network.target
[Service]
User=prometheus
Group=prometheus
ExecStart=/usr/sbin/blackbox_exporter  \
         --config.file=/etc/blackbox/blackbox.yml
[Install]
WantedBy=multi-user.target

systemctl enable blackbox_exporter.service
systemctl start blackbox_exporter.service
解读：prometheus通过定义blackbox_exporter的tcp_connect进行端口探测，实现原理通过类似于telnet port的方式
prometheus定义job配置：

  - job_name: 'port_status'
    metrics_path: /probe
    params:
      module: [tcp_connect]
    static_configs:
    - targets: ['172.16.60.1:2181','172.16.60.1:61616','172.16.60.1:11211']
      labels:
        instance: '172.16.60.1'
        tags: '自定义产品或者服务名'
    - targets: ['172.16.60.1:26004']
      labels:
        instance: '172.16.60.1'
        tags: 'service_name'
      relabel_configs:
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: 192.168.11.178:9115

3、加入规则文件（rules）

groups:
- name: blackbox_network_stats
  rules:
  - alert: 'url服务探测失败'
    expr: probe_success == 0
    for: 60s
    labels:
      severity: high
      alertinfo: push_blackbox_alert
    annotations:
      summary: "{{ $labels.instance }}探测失败"
      description: "url探测失败，请检查业务是否正常!!!"

4、重启prometheus并导入grafana json。最后实现效果图（导入json详见grafana篇）