需求
当前php-fpm进程是不是不够了,一通命令一打,也只能看当前情况。
熟话说无监控无度量就无诊断。
市面上有成熟的php-fpm status状态采集，展示工具那就是elasti stack全家桶
目标采集分析php-fpm status 状态数据
选型
Elastic公司全家桶之Beats(PHP-FPM Module@Metricbeat) + Elasticsearch + Kibana
Metricbeat 是 Beats 的一个采集组件
PHP-FPM Module 是 Metricbeat 的一个采集模块
整体的数据流方式
方式1 : Beats (Metricbeat )>Elasticsearch>Kibana
方式2: Beats (Metricbeat )>Logstash>[直连,redis队列,Kafka队列]>Elasticsearch>Kibana
先讲方式1
配置php-fpm
php-fpm.conf
[www]
...
pm.status_path = /phpfpm-status-www
[u]
...
pm.status_path = /phpfpm-status-u
[www] [u] 是独立的php-fpm pool 进程池,每个池子的pm.status_path是需要单独设置的
测试配置文件是否正确
/usr/local/php/sbin/php-fpm -t
[29-Mar-2017 16:15:08] NOTICE: configuration file /usr/local/php/etc/php-fpm.conf test is successful
重启php-fpm
/etc/init.d/php-fpm reload
Reload service php-fpm done
配置nginx
phpfpm.conf
server
{
listen 80;
server_name localhost;
location ~ ^/(phpfpm-status-www|phpstatuswww)$
{
fastcgi_pass unix:/tmp/php-cgi.sock;
include fastcgi.conf;
fastcgi_param SCRIPT_FILENAME $fastcgi_script_name;
}
location ~ ^/(phpfpm-status-u|phpstatusu)$
{
fastcgi_pass unix:/tmp/u-php-cgi.sock;
include fastcgi.conf;
fastcgi_param SCRIPT_FILENAME $fastcgi_script_name;
}
}
只给本地访问,这里监听的是localhost,也可用nginx allow deny的方式
测试配置文件是否正确
/usr/local/nginx/sbin/nginx -t
nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful
重启nginx
/etc/init.d/nginx reload
Reloading nginx daemon configuration....
用curl测试php-fpm status状态是否可以获取
curl http://localhost/phpfpm-status-www
pool: www
process manager: static
start time: 29/Mar/2017:16:19:09 +0800
start since: 75
accepted conn: 900
listen queue: 0
max listen queue: 0
listen queue len: 0
idle processes: 696
active processes: 4
total processes: 700
max active processes: 60
max children reached: 0
slow requests: 1
配置Metricbeat
Metricbeat Reference [5.3] » Modules » PHP-FPM Module
/etc/metricbeat/metricbeat.yml
#========================== Modules configuration ============================
metricbeat.modules:
#------------------------------- PHP-FPM Module -------------------------------
- module: php_fpm
metricsets: ["pool"]
enabled: true
period: 10s
status_path: "/phpfpm-status-www"
hosts: ["localhost:80"]
- module: php_fpm
metricsets: ["pool"]
enabled: true
period: 10s
status_path: "/phpfpm-status-u"
hosts: ["localhost:80"]
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["localhost:9200"]
测试配置文件是否正确
/usr/share/metricbeat/bin/metricbeat -configtest
Config OK
给metricbeat创建Elasticsearch的Index Template
Loading the Index Template in Elasticsearch
curl -XPUT 'http://localhost:9200/_template/metricbeat' -d@/etc/metricbeat/metricbeat.template.json
给metricbeat创建Kibana Dashboards
Loading Sample Kibana Dashboards
./scripts/import_dashboards -es http://localhost:9200
metricbeat中phpfpm模块，没有elastic官方的 Sample Kibana Dashboard
所以需要分析什么数据的自己在Kibana中自己创建visualize然后做成Dashboard
启动metricbeat
/etc/init.d/metricbeat start
配置Kibana
Management / Kibana / Indices > Add New
Index name or pattern 填写 “metricbeat-*”
Time-field name 选 “@timestamp”
分享我做的 metricbeat phpfpm kibana dashboard （不完整）
export.json
[
{
"_id": "PHP-FPM导航",
"_type": "visualization",
"_source": {
"title": "PHPFPM导航",
"visState": "{\n \"title\": \"PHP-FPM导航\",\n \"type\": \"markdown\",\n \"params\": {\n \"markdown\": \"- [Overview](#/dashboard/Metricbeat-phpfpm-overview)\\n\\n\\n```\\npool：php-fpm池的名称，一般都是应该是www\\nprocess manage：进程的管理方法，php-fpm支持三种管理方法，分别是static,dynamic和ondemand，一般情况下都是dynamic\\nstart time：php-fpm启动时候的时间，不管是restart或者reload都会更新这里的时间\\nstart since：php-fpm自启动起来经过的时间，默认为秒\\naccepted conn：当前接收的连接数\\nlisten queue：在队列中等待连接的请求个数，如果这个数字为非0，那么最好增加进程的fpm个数\\nmax listen queue：从fpm启动以来，在队列中等待连接请求的最大值\\nlisten queue len：等待连接的套接字队列大小\\nidle processes：空闲的进程个数\\nactive processes：活动的进程个数\\ntotal processes：总共的进程个数\\nmax active processes：从fpm启动以来，活动进程的最大个数，如果这个值小于当前的max_children，可以调小此值\\nmax children reached：当pm尝试启动更多的进程，却因为max_children的限制，没有启动更多进程的次数。如果这个值非0，那么可以适当增加fpm的进程数\\nslow requests：慢请求的次数，一般如果这个值未非0，那么可能会有慢的php进程，一般一个不好的mysql查询是最大的祸首。\\n```\"\n },\n \"aggs\": [],\n \"listeners\": {}\n}",
"uiStateJSON": "{}",
"description": "",
"version": 1,
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{\n \"query\": {\n \"query_string\": {\n \"query\": \"*\",\n \"analyze_wildcard\": true\n }\n },\n \"filter\": []\n}"
}
}
},
{
"_id": "PHPFPM-processes-active",
"_type": "visualization",
"_source": {
"title": "PHPFPM processes active",
"visState": "{\"title\":\"PHPFPM processes active\",\"type\":\"area\",\"params\":{\"shareYAxis\":true,\"addTooltip\":true,\"addLegend\":true,\"legendPosition\":\"right\",\"smoothLines\":false,\"scale\":\"linear\",\"interpolate\":\"linear\",\"mode\":\"stacked\",\"times\":[],\"addTimeMarker\":false,\"defaultYExtents\":false,\"setYExtents\":false,\"yAxis\":{}},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"avg\",\"schema\":\"metric\",\"params\":{\"field\":\"php_fpm.pool.processes.active\",\"customLabel\":\"活动\"}},{\"id\":\"2\",\"enabled\":true,\"type\":\"date_histogram\",\"schema\":\"segment\",\"params\":{\"field\":\"@timestamp\",\"interval\":\"auto\",\"customInterval\":\"2h\",\"min_doc_count\":1,\"extended_bounds\":{},\"customLabel\":\"时间\"}},{\"id\":\"3\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"group\",\"params\":{\"field\":\"php_fpm.pool.name\",\"size\":5,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"pool池\"}},{\"id\":\"4\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"split\",\"params\":{\"field\":\"beat.name\",\"size\":5,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"主机\",\"row\":true}}],\"listeners\":{}}",
"uiStateJSON": "{}",
"description": "",
"version": 1,
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{\"index\":\"metricbeat-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
}
}
},
{
"_id": "PHPFPM-connections-queued",
"_type": "visualization",
"_source": {
"title": "PHPFPM connections queued",
"visState": "{\"title\":\"PHPFPM connections queued\",\"type\":\"area\",\"params\":{\"shareYAxis\":true,\"addTooltip\":true,\"addLegend\":true,\"legendPosition\":\"right\",\"smoothLines\":false,\"scale\":\"linear\",\"interpolate\":\"linear\",\"mode\":\"stacked\",\"times\":[],\"addTimeMarker\":false,\"defaultYExtents\":false,\"setYExtents\":false,\"yAxis\":{}},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"avg\",\"schema\":\"metric\",\"params\":{\"field\":\"php_fpm.pool.connections.queued\",\"customLabel\":\"队列\"}},{\"id\":\"2\",\"enabled\":true,\"type\":\"date_histogram\",\"schema\":\"segment\",\"params\":{\"field\":\"@timestamp\",\"interval\":\"auto\",\"customInterval\":\"2h\",\"min_doc_count\":1,\"extended_bounds\":{},\"customLabel\":\"时间\"}},{\"id\":\"3\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"group\",\"params\":{\"field\":\"php_fpm.pool.name\",\"size\":5,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"pool池\"}},{\"id\":\"4\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"split\",\"params\":{\"field\":\"beat.name\",\"size\":5,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"主机\",\"row\":true}}],\"listeners\":{}}",
"uiStateJSON": "{}",
"description": "",
"version": 1,
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{\"index\":\"metricbeat-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
}
}
},
{
"_id": "PHPFPM-processes-idle",
"_type": "visualization",
"_source": {
"title": "PHPFPM processes idle",
"visState": "{\"title\":\"PHPFPM processes idle\",\"type\":\"area\",\"params\":{\"shareYAxis\":true,\"addTooltip\":true,\"addLegend\":true,\"legendPosition\":\"right\",\"smoothLines\":false,\"scale\":\"linear\",\"interpolate\":\"linear\",\"mode\":\"stacked\",\"times\":[],\"addTimeMarker\":false,\"defaultYExtents\":false,\"setYExtents\":false,\"yAxis\":{}},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"avg\",\"schema\":\"metric\",\"params\":{\"field\":\"php_fpm.pool.processes.idle\",\"customLabel\":\"空闲\"}},{\"id\":\"2\",\"enabled\":true,\"type\":\"date_histogram\",\"schema\":\"segment\",\"params\":{\"field\":\"@timestamp\",\"interval\":\"auto\",\"customInterval\":\"2h\",\"min_doc_count\":1,\"extended_bounds\":{},\"customLabel\":\"时间\"}},{\"id\":\"4\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"group\",\"params\":{\"field\":\"php_fpm.pool.name\",\"size\":5,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"pool池\"}},{\"id\":\"3\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"split\",\"params\":{\"field\":\"beat.name\",\"size\":5,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"主机\",\"row\":true}}],\"listeners\":{}}",
"uiStateJSON": "{}",
"description": "",
"version": 1,
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{\"index\":\"metricbeat-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
}
}
},
{
"_id": "PHPFPM-slow_requests",
"_type": "visualization",
"_source": {
"title": "PHPFPM slow_requests",
"visState": "{\"title\":\"PHPFPM slow_requests\",\"type\":\"area\",\"params\":{\"shareYAxis\":true,\"addTooltip\":true,\"addLegend\":true,\"legendPosition\":\"right\",\"smoothLines\":false,\"scale\":\"linear\",\"interpolate\":\"linear\",\"mode\":\"stacked\",\"times\":[],\"addTimeMarker\":false,\"defaultYExtents\":false,\"setYExtents\":false,\"yAxis\":{}},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"avg\",\"schema\":\"metric\",\"params\":{\"field\":\"php_fpm.pool.slow_requests\"}},{\"id\":\"2\",\"enabled\":true,\"type\":\"date_histogram\",\"schema\":\"segment\",\"params\":{\"field\":\"@timestamp\",\"interval\":\"auto\",\"customInterval\":\"2h\",\"min_doc_count\":1,\"extended_bounds\":{},\"customLabel\":\"时间\"}},{\"id\":\"4\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"group\",\"params\":{\"field\":\"php_fpm.pool.name\",\"size\":5,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"pool池\"}},{\"id\":\"3\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"split\",\"params\":{\"field\":\"beat.name\",\"size\":5,\"order\":\"desc\",\"orderBy\":\"1\",\"customLabel\":\"主机\",\"row\":true}}],\"listeners\":{}}",
"uiStateJSON": "{}",
"description": "",
"version": 1,
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{\"index\":\"metricbeat-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"
}
}
}
]
补充logstash采集数据（方式2）
配置Metricbeat
/etc/metricbeat/metricbeat.yml
#================================ Outputs =====================================
# The Logstash hosts
output.logstash:
hosts: ["localhost:5044"]
output输出到logstash的input tcp 5044上
input输入参考上面方式1 Metricbeat 的配置方法
配置logstash2.x
Logstash Reference [2.4] » Input plugins » beats
/etc/logstash/conf.d/beats.conf
input {
beats {
host => "127.0.0.1"
port => 5044
}
}
filter {
}
output {
... 输出到kafka 或者reids 或者 elasticsearch等,具体配置方法看官方文档
}
input输入tcp 5044 监听本机
output输出，根据自己环境来
测试配置文件是否正确
/opt/logstash/bin/logstash -t
Configuration OK
启动logstash
/etc/init.d/logstash start
提示
- 使用测试配置文件的功能测试配置文件是否书写正确
- 使用tail -f 日志的方式查错
- 注意tcp监听的安全问题，别暴露到公网IP上