With a little help from rastro in the #logstash IRC channel I got php to to log in to json and logstash to parse out the timestamps. Like this:
{ "mem": 786432, "sec": 0.001, "cpu": 0, "why": "/test.php?test=4", "what": "/var/www/test.php", "who": "-", "when": "1454100567" }
Set the following in /var/log/php5-fpm/www.access.log:
access.log = /var/log/php5-fpm/$pool.access.log
access.format = '{"when":"%{%s}t","who":"%{HTTP_X_FORWARDED_FOR}e","what":"%f","why":"%{REQUEST_URI}e","cpu":%C,"sec":%d,"mem":%M}'
Make the target dir and test:
mkdir /var/log/php5-fpm
service php5-fpm restart
curl -s "http://localhost/test.php?test=4"
You'll see logs like the following:
$ tail -1 /var/log/php5-fpm/www.access.log | python -m json.tool
{
"mem": 786432,
"sec": 0.001,
"cpu": 0,
"why": "/test.php?test=4",
"what": "/var/www/test.php",
"who": "-",
"when": "1454100567"
}
NOTE: The 'who' field will get X-Forwarded-For headers in production.
Now to parse this in logstash:
filter {
if [type] == "php-access" {
json {
source => "message"
target => "fields"
}
date {
match => [ "[fields][when]", "UNIX" ]
remove_field => [ "[fields][when]", "message" ]
}
mutate {
convert => {
"[fields][cpu]" => "float"
"[fields][sec]" => "float"
"[fields][mem]" => "integer"
}
}
}
}
Voila! You'll see output like:
{
"@version" => "1",
"@timestamp" => "2016-01-29T20:19:38.000Z",
"beat" => {
"hostname" => "vagrant",
"name" => "vagrant"
},
"count" => 1,
"fields" => {
"who" => "-",
"what" => "/var/www/test.php",
"why" => "/test.php?test=4",
"cpu" => 0.0,
"sec" => 0.004,
"mem" => 786432
},
"input_type" => "log",
"offset" => 4849,
"source" => "/var/log/php5-fpm/www.access.log",
"tags" => [
[0] "vagrant",
[1] "dev"
],
"type" => "php-access"
}
comments powered by Disqus