[wellylug] Apache (and other) log files to database
Cliff Pratt
enkidu at cliffp.com
Wed Dec 6 22:11:42 NZDT 2006
michael at diaspora.gen.nz wrote:
> Cliff Pratt writes:
>> I have a need to write Apache (and maybe other) logfiles to a database.
>
> pglogd logs to a Postgres database by reading Apache common logging
> format records from a Unix socket.
>
> If you're developing a web application, rather than a generic "I'm
> running a web server", you may want to write a custom logger to add
> "interesting information" from your application; in which case you'll
> want a custom schema, etc. anyway, and you'd might be better off just
> writing a Perl program.
>
> Watch out for delaying web performance by making threads block logging
> into the db; it can often be better to write logs into a file, and
> then load a database from the flat file through a bulk loader utility
> (eg Postgres' COPY FROM routine).
> -- michael.
>
Ah, I'm not wanting to write to the DB in real time. I'm reading and
splitting and writing logfiles which have been copied to a central
server. Copy happens once a day.
I think I'll persist with Perl.
Incidentally pglogd looks interesting, but it appears not to work with
PostgreSQL 8.1 from the web page. I like the 'write to FIFO' technique
and I may play with that for something else.
Cheers,
Cliff
More information about the wellylug
mailing list