[wellylug] Apache (and other) log files to database

[Cliff Pratt]

andrej at paradise.net.nz wrote:
>>> And what stops you from using syslog for apache?
>> You imply that nothing does! I've looked at the docs and it appears to 
>> me that I can use syslog-ng as a filter (file in and file out or out to 
>> a program), but it looks like I'll have to write a program to split up 
>> the records and write them to the DB. I might just as well process the 
>> files directly. Maybe I haven't got to the right bit of the docs yet!
> It's not nothing, but it can be done;  the benefit is that you
> wouldn't store the files twice - reduced overhead in both
> processing and storage ... an interesing approach is described
> at
> http://www.oreillynet.com/pub/a/sysadmin/2006/10/12/httpd-syslog.html
> 
> If you have syslog-ng doing the script method as well you MAY,
> as you pointed out, as well do the parsing and processing yourself,
> but there's still the extra (doubled) disk-writes :}
> 
Most of the logs are copied to a central server. I'm not too worried 
about the double write problem!

Cheers,

Cliff