[wellylug] Cron + scripting

Mon Apr 11 08:23:40 NZST 2005

I think most of the reason that it crashes is that I have to much stuff
running on it (ie: spamassassin, clamav, fetchmail.

The system it is running is SME 6.0.1 (2.4.20-18.7). It is the stock kernel
that comes with SME Server 6.0.1.

Here is what my logs say when the system crashes:
Apr 10 02:50:12 server kernel: Out of Memory: Killed process 1451 (httpd).
Apr 10 03:01:55 server kernel: Out of Memory: Killed process 1443 (httpd).
Apr 10 03:12:24 server kernel: Out of Memory: Killed process 1444 (httpd).
Apr 10 03:17:02 server kernel: Out of Memory: Killed process 1445 (httpd).
Apr 10 03:54:49 server kernel: Out of Memory: Killed process 3605 (httpd).
Apr 10 03:55:18 server kernel: Out of Memory: Killed process 1446 (httpd).
Apr 10 03:58:58 server kernel: Out of Memory: Killed process 3611 (httpd).
Apr 10 03:59:18 server kernel: Out of Memory: Killed process 3612 (httpd).
Apr 10 04:00:06 server kernel: Out of Memory: Killed process 1447 (httpd).
Apr 10 04:00:28 server kernel: Out of Memory: Killed process 1448 (httpd).
Apr 10 04:00:56 server kernel: Out of Memory: Killed process 1449 (httpd).
Apr 10 04:01:41 server kernel: Out of Memory: Killed process 1450 (httpd).
Apr 10 04:01:42 server kernel: Out of Memory: Killed process 1452 (httpd).
Apr 10 04:02:18 server kernel: Out of Memory: Killed process 2991 (httpd).
Apr 10 04:02:34 server kernel: Out of Memory: Killed process 1820 (mysqld).
Apr 10 04:02:34 server kernel: Out of Memory: Killed process 1954 (mysqld).
Apr 10 04:02:34 server kernel: Out of Memory: Killed process 1955 (mysqld).
Apr 10 04:02:34 server kernel: Out of Memory: Killed process 1957 (mysqld).
Apr 10 04:14:06 server kernel: Out of Memory: Killed process 3615 (httpd).
Apr 10 04:15:01 server kernel: Out of Memory: Killed process 3616 (httpd).
Apr 10 04:19:01 server kernel: Out of Memory: Killed process 3618 (httpd).
Apr 10 04:19:30 server kernel: Out of Memory: Killed process 3619 (httpd).
Apr 10 04:20:06 server kernel: Out of Memory: Killed process 3620 (httpd).
Apr 10 04:20:25 server kernel: Out of Memory: Killed process 3624 (httpd).
Apr 10 04:20:58 server kernel: Out of Memory: Killed process 3627 (httpd).
Apr 10 04:21:48 server kernel: Out of Memory: Killed process 3628 (httpd).
Apr 10 04:22:12 server kernel: Out of Memory: Killed process 3648 (httpd).
Apr 10 04:22:46 server last message repeated 6 times
Apr 10 04:23:55 server last message repeated 12 times
Apr 10 04:24:56 server last message repeated 13 times
Apr 10 04:26:01 server last message repeated 12 times
Apr 10 04:27:07 server last message repeated 11 times
Apr 10 04:28:08 server last message repeated 12 times
Apr 10 04:29:21 server last message repeated 2 times
Apr 10 04:31:03 server last message repeated 8 times
Apr 10 04:32:42 server last message repeated 3 times
Apr 10 04:34:09 server kernel: Out of Memory: Killed process 3648 (httpd).
Apr 10 04:35:58 server last message repeated 3 times
Apr 10 04:37:13 server last message repeated 2 times
Apr 10 04:38:14 server kernel: Out of Memory: Killed process 3648 (httpd).
Apr 10 04:39:52 server last message repeated 10 times
Apr 10 04:39:52 server kernel: Out of Memory: Killed process 3648 (httpd).
Apr 10 04:42:42 server last message repeated 8 times
Apr 10 04:44:08 server last message repeated 12 times
Apr 10 04:44:57 server kernel: Out of Memory: Killed process 3648 (httpd).
Apr 10 04:44:57 server kernel: Out of Memory: Killed process 3648 (httpd).
Apr 10 04:48:26 server last message repeated 14 times
Apr 10 04:49:04 server last message repeated 4 times
Apr 10 04:51:43 server last message repeated 7 times
Apr 10 04:52:03 server last message repeated 5 times
Apr 10 04:52:03 server kernel: Out of Memory: Killed process 3648 (httpd).
Apr 10 04:52:03 server kernel: Out of Memory: Killed process 3648 (httpd).
Apr 10 05:09:43 server last message repeated 92 times
Apr 10 05:09:43 server kernel: Out of Memory: Killed process 1963 (clamd).
Apr 10 05:09:43 server kernel: Out of Memory: Killed process 2023 (clamd).
Apr 10 05:25:43 server kernel: Out of Memory: Killed process 3667 (httpd).
Apr 10 05:26:01 server kernel: Out of Memory: Killed process 3650 (httpd).
Apr 10 05:26:08 server kernel: Out of Memory: Killed process 3651 (httpd).
Apr 10 05:26:31 server kernel: Out of Memory: Killed process 3652 (httpd).
Apr 10 05:26:43 server kernel: Out of Memory: Killed process 3654 (httpd).
Apr 10 05:27:11 server kernel: Out of Memory: Killed process 3655 (httpd).
Apr 10 05:27:30 server kernel: Out of Memory: Killed process 3658 (httpd).
Apr 10 05:27:43 server kernel: Out of Memory: Killed process 3649 (httpd).
Apr 10 05:28:08 server kernel: Out of Memory: Killed process 3696 (httpd).
Apr 10 05:28:45 server last message repeated 4 times
Apr 10 05:29:49 server last message repeated 12 times
Apr 10 05:30:51 server last message repeated 12 times
Apr 10 05:31:54 server last message repeated 11 times
Apr 10 05:32:55 server last message repeated 12 times
Apr 10 05:33:55 server last message repeated 11 times
Apr 10 05:34:59 server last message repeated 12 times
Apr 10 05:36:14 server last message repeated 10 times
Apr 10 05:37:23 server last message repeated 14 times
Apr 10 05:38:27 server last message repeated 12 times
Apr 10 05:39:40 server last message repeated 12 times
Apr 10 05:40:44 server last message repeated 13 times
Apr 10 05:41:49 server last message repeated 11 times
Apr 10 05:42:53 server last message repeated 13 times
Apr 10 05:43:59 server last message repeated 12 times
Apr 10 05:45:06 server last message repeated 13 times
Apr 10 05:46:10 server last message repeated 13 times
Apr 10 05:47:10 server last message repeated 10 times
Apr 10 05:48:15 server last message repeated 15 times
Apr 10 05:49:27 server last message repeated 9 times
Apr 10 05:50:34 server last message repeated 16 times
Apr 10 05:51:40 server last message repeated 14 times
Apr 10 05:52:40 server last message repeated 10 times
Apr 10 05:54:22 server last message repeated 7 times
Apr 10 05:55:18 server last message repeated 13 times
Apr 10 05:56:51 server last message repeated 32 times
Apr 10 05:57:12 server kernel: Out of Memory: Killed process 3696 (httpd).
Apr 10 05:57:12 server kernel: Out of Memory: Killed process 3696 (httpd).
Apr 10 05:57:12 server kernel: Out of Memory: Killed process 3696 (httpd).
Apr 10 06:23:11 server last message repeated 131 times
Apr 10 06:23:11 server kernel: Out of Memory: Killed process 1315 (slapd).
Apr 10 06:23:11 server kernel: Out of Memory: Killed process 1316 (slapd).
Apr 10 06:23:12 server kernel: Out of Memory: Killed process 1317 (slapd).
Apr 10 06:23:21 server kernel: Out of Memory: Killed process 3711 (httpd).
Apr 10 06:23:53 server kernel: Out of Memory: Killed process 3711 (httpd).
Apr 10 06:25:10 server last message repeated 6 times
Apr 10 06:26:17 server last message repeated 8 times
Apr 10 06:27:20 server last message repeated 5 times
Apr 10 06:28:24 server last message repeated 10 times
Apr 10 06:49:26 server last message repeated 5 times
Apr 10 07:01:10 server kernel: Out of Memory: Killed process 3719 (httpd).
Apr 10 07:01:22 server kernel: Out of Memory: Killed process 3700 (httpd).
Apr 10 07:01:37 server kernel: Out of Memory: Killed process 3706 (httpd).
Apr 10 07:01:59 server kernel: Out of Memory: Killed process 3701 (httpd).
Apr 10 07:02:21 server kernel: Out of Memory: Killed process 3702 (httpd).
Apr 10 07:02:32 server kernel: Out of Memory: Killed process 3703 (httpd).
Apr 10 07:02:47 server kernel: Out of Memory: Killed process 3704 (httpd).
Apr 10 07:02:59 server kernel: Out of Memory: Killed process 3705 (httpd).
Apr 10 07:03:31 server kernel: Out of Memory: Killed process 3722 (httpd).
Apr 10 07:04:01 server last message repeated 6 times
Apr 10 07:05:03 server last message repeated 11 times
Apr 10 07:06:08 server last message repeated 14 times
Apr 10 07:07:12 server last message repeated 5 times
Apr 10 07:08:32 server last message repeated 15 times
Apr 10 07:09:43 server last message repeated 15 times
Apr 10 07:10:49 server last message repeated 20 times
Apr 10 07:13:57 server last message repeated 8 times
Apr 10 07:13:57 server kernel: Out of Memory: Killed process 3722 (httpd).
Apr 10 07:16:43 server last message repeated 12 times
Apr 10 07:17:01 server kernel: Out of Memory: Killed process 3722 (httpd).
Apr 10 07:17:01 server kernel: Out of Memory: Killed process 3722 (httpd).
Apr 10 07:17:01 server kernel: Out of Memory: Killed process 3722 (httpd).
Apr 10 07:27:17 server last message repeated 21 times
Apr 10 07:28:50 server last message repeated 6 times
Apr 10 07:29:07 server kernel: Out of Memory: Killed process 3722 (httpd).
Apr 10 07:30:57 server kernel: Out of Memory: Killed process 3722 (httpd).
Apr 10 07:34:09 server last message repeated 2 times
Apr 10 07:36:14 server last message repeated 6 times
Apr 10 07:39:05 server last message repeated 3 times
Apr 10 07:39:05 server kernel: Out of Memory: Killed process 3722 (httpd).
Apr 10 07:41:15 server last message repeated 9 times
Apr 10 07:42:49 server kernel: Out of Memory: Killed process 3722 (httpd).
Apr 10 07:44:08 server last message repeated 3 times
Apr 10 07:44:47 server kernel: Out of Memory: Killed process 3722 (httpd).
Apr 10 07:45:27 server kernel: Out of Memory: Killed process 3722 (httpd).
Apr 10 07:46:10 server kernel: Out of Memory: Killed process 3722 (httpd).
Apr 10 07:46:27 server kernel: Out of Memory: Killed process 3722 (httpd).
Apr 10 07:46:31 server kernel: Out of Memory: Killed process 3722 (httpd).
Apr 10 07:46:39 server kernel: Out of Memory: Killed process 3722 (httpd).
Apr 10 07:47:32 server kernel: Out of Memory: Killed process 3722 (httpd).
Apr 10 07:47:32 server kernel: Out of Memory: Killed process 3722 (httpd).
Apr 10 07:48:07 server kernel: Out of Memory: Killed process 3722 (httpd).
Apr 10 07:48:14 server kernel: Out of Memory: Killed process 3722 (httpd).
Apr 10 07:48:36 server kernel: Out of Memory: Killed process 3722 (httpd).
Apr 10 08:46:41 server last message repeated 9 times
Apr 10 09:35:28 server kernel: ss 3725 (httpd).
Apr 10 09:35:28 server kernel: Out of Memory: Killed process 3725 (httpd).
Apr 10 09:35:28 server kernel: Out of Memory: Killed process 3725 (httpd).
Apr 10 09:37:19 server last message repeated 12 times
Apr 10 09:38:05 server kernel: Out of Memory: Killed process 3747 (httpd).
Apr 10 09:38:07 server kernel: Out of Memory: Killed process 3747 (httpd).
Apr 10 09:39:40 server last message repeated 34 times
Apr 10 09:39:40 server kernel: Out of Memory: Killed process 3725 (httpd).
Apr 10 09:39:40 server kernel: Out of Memory: Killed process 3747 (httpd).
Apr 10 09:39:40 server kernel: Out of Memory: Killed process 3747 (httpd).
Apr 10 09:39:41 server kernel: Out of Memory: Killed process 3747 (httpd).

When the system does die, the hard-drive starts going nuts and is doing
something (I would guess trying to swap, but with no memory to). I only
128MB RAM in the system, but I have a client that is running the exact same
setup but with 1GB RAM and they have no problems, so I'm guessing I need
more RAM which is why I'm going to upgrade the machine next month.

Michael

-----Original Message-----
From: wellylug-admin at lists.wellylug.org.nz
[mailto:wellylug-admin at lists.wellylug.org.nz] On Behalf Of Jamie Baddeley
Sent: Sunday, April 10, 2005 10:37 PM
To: wellylug at lists.wellylug.org.nz
Subject: RE: [wellylug] Cron + scripting

On Sun, 2005-04-10 at 21:21 +1200, Michael Dittmer (An Inside Job)
wrote:
> The reason I need to reboot the server is that after a few days it 
> seems to run out of memory and crashes. If I reboot the machine at 1am 
> everyday, then this will stop it from crashing.

Eww.

>  I will be adding more RAM to it sometimes next month.

Which will just delay the inevitable. 

In the interests of making sure that Linux based systems have better uptime
than Windows based systems, I'd be prepared to help with trying to solve the
problem.

Have you any idea where and why you're leaking memory? what processes is
this server running to do that? What diagnostics have you run? what does top
tell you?
Kernel version? Describe the crash please?

jamie

> 
> Michael
> 
> -----Original Message-----
> From: wellylug-admin at lists.wellylug.org.nz
> [mailto:wellylug-admin at lists.wellylug.org.nz] On Behalf Of Jamie 
> Baddeley
> Sent: Sunday, April 10, 2005 8:59 PM
> To: wellylug at lists.wellylug.org.nz
> Subject: RE: [wellylug] Cron + scripting
> 
> can I ask why on earth you need to reboot the machine? That sounds 
> like a technique borrowed from the 'ctrl-alt-del' school of computing, 
> and should be something that is avoidable on linux.
> 
> jamie
> 
> On Sun, 2005-04-10 at 20:51 +1200, Michael Dittmer (An Inside Job)
> wrote:
> > Hi Wayne
> > 
> > I'm running SME Server 6.0.1.
> > 
> > I'll take a look at man cron.
> > 
> > Michael
> > 
> > -----Original Message-----
> > From: wellylug-admin at lists.wellylug.org.nz
> > [mailto:wellylug-admin at lists.wellylug.org.nz] On Behalf Of Wayne 
> > Koorts
> > Sent: Sunday, April 10, 2005 8:33 PM
> > To: wellylug at lists.wellylug.org.nz
> > Subject: Re: [wellylug] Cron + scripting
> > 
> > > How do I write a script to do this and also how do I get cron to 
> > > run the script?
> > 
> > What OS is running on your file server?
> > 
> > 'man cron' should give you information on how to have cron execute
> scripts.
> > 
> > --
> > Regards,
> > Wayne Koorts
> > Registered Linux User #330079
> > www.wkoorts.com
> > 
> > 
> > --
> > No virus found in this outgoing message.
> > Checked by AVG Anti-Virus.
> > Version: 7.0.300 / Virus Database: 266.9.5 - Release Date: 4/7/2005
> > 
> > 
> > --
> > Wellington Linux Users Group Mailing List: 
> > wellylug at lists.wellylug.org.nz To Leave:  
> > http://lists.wellylug.org.nz/mailman/listinfo/wellylug
> > 
> > 
> > 
> 
> --
> Jamie Baddeley <wellylug at vpc.co.nz>
> 
> 
> --
> Wellington Linux Users Group Mailing List: 
> wellylug at lists.wellylug.org.nz To Leave:  
> http://lists.wellylug.org.nz/mailman/listinfo/wellylug
> 
> 

--
Jamie Baddeley <wellylug at vpc.co.nz>

-- 
Wellington Linux Users Group Mailing List: wellylug at lists.wellylug.org.nz
To Leave:  http://lists.wellylug.org.nz/mailman/listinfo/wellylug