[wellylug] Apache (and other) log files to database

Cliff Pratt enkidu at cliffp.com
Wed Dec 6 22:11:42 NZDT 2006


michael at diaspora.gen.nz wrote:
> Cliff Pratt writes:
>> I have a need to write Apache (and maybe other) logfiles to a database.
> 
> pglogd logs to a Postgres database by reading Apache common logging
> format records from a Unix socket.
> 
> If you're developing a web application, rather than a generic "I'm
> running a web server", you may want to write a custom logger to add
> "interesting information" from your application; in which case you'll
> want a custom schema, etc. anyway, and you'd might be better off just
> writing a Perl program.
> 
> Watch out for delaying web performance by making threads block logging
> into the db; it can often be better to write logs into a file, and
> then load a database from the flat file through a bulk loader utility
> (eg Postgres' COPY FROM routine).
>     -- michael.
> 
Ah, I'm not wanting to write to the DB in real time. I'm reading and 
splitting and writing logfiles which have been copied to a central 
server. Copy happens once a day.

I think I'll persist with Perl.

Incidentally pglogd looks interesting, but it appears not to work with 
PostgreSQL 8.1 from the web page. I like the 'write to FIFO' technique 
and I may play with that for something else.

Cheers,

Cliff




More information about the wellylug mailing list