Dan
Published

Fri 29 January 2016

←Home

How to log php requests to logstash in json

With a little help from rastro in the #logstash IRC channel I got php to to log in to json and logstash to parse out the timestamps. Like this:

{ "mem": 786432, "sec": 0.001, "cpu": 0, "why": "/test.php?test=4", "what": "/var/www/test.php", "who": "-", "when": "1454100567" }

Set the following in /var/log/php5-fpm/www.access.log:

access.log = /var/log/php5-fpm/$pool.access.log

access.format = '{"when":"%{%s}t","who":"%{HTTP_X_FORWARDED_FOR}e","what":"%f","why":"%{REQUEST_URI}e","cpu":%C,"sec":%d,"mem":%M}'

Make the target dir and test:

mkdir /var/log/php5-fpm
service php5-fpm restart
curl -s "http://localhost/test.php?test=4"

You'll see logs like the following:

$ tail -1 /var/log/php5-fpm/www.access.log  | python -m json.tool
{
  "mem": 786432,
  "sec": 0.001,
  "cpu": 0,
  "why": "/test.php?test=4",
  "what": "/var/www/test.php",
  "who": "-",
  "when": "1454100567"
}

NOTE: The 'who' field will get X-Forwarded-For headers in production.

Now to parse this in logstash:

filter {
    if [type] == "php-access" {
        json {
            source => "message"
            target => "fields"
        }
        date {
            match => [ "[fields][when]", "UNIX" ]
            remove_field => [ "[fields][when]", "message" ]
        }
        mutate {
            convert => {
                "[fields][cpu]" => "float"
                "[fields][sec]" => "float"
                "[fields][mem]" => "integer"
            }
        }
    }
}

Voila! You'll see output like:

{
      "@version" => "1",
    "@timestamp" => "2016-01-29T20:19:38.000Z",
          "beat" => {
        "hostname" => "vagrant",
            "name" => "vagrant"
    },
         "count" => 1,
        "fields" => {
         "who" => "-",
        "what" => "/var/www/test.php",
         "why" => "/test.php?test=4",
         "cpu" => 0.0,
         "sec" => 0.004,
         "mem" => 786432
    },
    "input_type" => "log",
        "offset" => 4849,
        "source" => "/var/log/php5-fpm/www.access.log",
          "tags" => [
        [0] "vagrant",
        [1] "dev"
    ],
          "type" => "php-access"
}
Go Top
comments powered by Disqus