prometheus监控应用及设置告警规则配置方法

1.prometheus监控haproxy安装haproxyyum install haproxy -y编辑haproxy配置文件vim /etc/haproxy/haproxy.cfgglobalmaxconn 100000chroot /usr/local/haproxyuid 99gid 99daemonnbproc 1pidfile /usr/local/haproxy/logs/haproxy.pidlog 127.0.0.1 local3 infodefaultsoption http-keep-alivemaxconn 100000mode httptimeout connect 5000mstimeout client 50000mstimeout server 50000mslisten stats #haproxy状态页面监听地址mode httpbind 0.0.0.0:8888frontend frontend_www_example_combind 10.0.0.150:88mode httpoption httploglog global default_backend backend_www_example_combackend backend_www_example_comoption forwardfor header X-REAL-IPoption httpchk HEAD / HTTP/1.0balance sourceserver web-node1 10.0.0.150:8080 check inter 2000 rise 30 fall 15根据haproxy配置文件中设置的haproxy根目录地址创建haproxy根目录mkdir -p /usr/local/haproxy启动haproxysystemctl start haproxy下载安装haproxy_exporter地址cd /usr/local/src/rz haproxy_exporter-0.9.0.linux-386.tar.gztar xf haproxy_exporter-0.9.0.linux-386.tar.gzcd haproxy_exporter-0.9.0.linux-386把haproxy_exporter文件复制到/usr/bin目录下cp haproxy_exporter /usr/bin启动haproxy_exporterhaproxy_exporter -haproxy.scrape-uri="http:/10.0.0.150:8888/baz?stats;csv" -web.listen-address="10.0.0.150:9100" &#-haproxy.scrape-uri是haproxy状态信息采集页面地址端口号和配置文件状态页面监听端口号相同状态，-web.listen-address是haproxy服务监听地址在prometheus配置文件中配置监控haproxy实例vi /etc/prometheus/prometheus.yml - job_name: haproxy #添加一个监控job名称 scrape_interval: 5s #监控采集数据时间间隔 scrape_timeout: 5s #监控采集数据超时时间 metrics_path: /metrics scheme: http static_configs: #监控静态目标配置 - targets: - 10.0.0.150:9100 labels: #监控静态目标标签 instance: haproxy:9100 重启prometheussystemctl restart prometheus刷新prometheus的target页面看到了prometheus监控haproxy的目标登录grafana创建haproxy数据源选择haproxy数据源导入haproxy监控大屏模板，模板地址2.prometheus监控nginx，参考下载nginx-module-vts模块cd /usr/local/src/git clone 重新编译nginx安装指定nginx-module-vts模块路径cd /usr/local/src/nginx-1.16.1./configure -prefix=/usr/local/nginx-1.16.1/ -with-http_ssl_module -with-http_stub_status_module -add-module=/usr/local/src/nginx-module-vts/ -user=www -group=www 在nginx配置文件server标签中添加status状态页面,http标签中添加vhost_traffic_status_zonevim /usr/local/nginx/conf/nginx.confserver location /status vhost_traffic_status_display; vhost_traffic_status_display_format html; vhost_traffic_status_filter_by_set_key $uri uri:$server_name; #每个uri访问量 vhost_traffic_status_filter_by_set_key $status $server_name; #http code统计 vhost_traffic_status_filter_by_set_key $upstream_addr upstream:backend; #后端转发统计 vhost_traffic_status_filter_by_set $remote_port client:ports:$server_name; #请求端口统计 vhost_traffic_status_filter_by_set_key $remote_addr client:addr:$server_name; #请求IP统计http include mime.types; default_type application/octet-stream;vhost_traffic_status_zone;启动nginx/usr/local/nginx/sbin/nginx打开浏览器输入宿主机ip/status访问nginx状态页面安装nginx-exporter，下载地址cd /usr/local/src/rz nginx-vts-exporter-0.9.1.linux-amd64.tar.gztar xf nginx-vts-exporter-0.9.1.linux-amd64.tar.gzmv nginx-vts-exporter-0.9.1.linux-amd64 /usr/local/nginx-vts-exporter启动nginx-vts-exporter服务，如果需要采集不同的nginx监控信息，编写不同收集的监控信息的ip地址的nginx-vts-exporter服务名称vim /usr/lib/systemd/system/nginx_vts_exporter_172.16.30.23.serviceUnitDescription=prometheus_nginx_vtsAfter=network.targetServiceType=simpleExecStart=/usr/local/nginx-vts-exporter/nginx-vts-exporter -nginx.scrape_uri=http:/172.16.20.33/status/format/json -telemetry.address 172.16.2.56:9913# 如果有多个nginx监控采集页面需要收集，nginx.scrape_uri和telemetry.address都要不同Restart=on-failureInstallWantedBy=multi-user.target# nginx.scrape_uri是nginx状态服务采集信息页面地址启动nginx_vts_exportersystemctl daemon-reloadsystemctl start nginx_vts_exporternetstat -ltunp | grep nginx-vtstcp6 0 0 :9913 :* LISTEN 104725/nginx-vts-ex修改prometheus配置文件添加nginx监控节点- job_name: 'nginx' # 添加nginx监控节点配置 static_configs: - targets: - '10.0.0.150:9913' #nginx监控节点配置nginx_vts_exporter端口号为9913重启prometheussystemctl restart prometheus刷新prometheus监控target页面看到nginx监控节点登录grafana添加nginx监控数据源导入grafana模板nginx监控指标说明n