[wellylug] diskless headless

David Antliff david.antliff at gmail.com
Thu Nov 4 15:18:58 NZDT 2010


On Thu, Nov 4, 2010 at 13:36, E Chalaron <e.chalaron at xtra.co.nz> wrote:
> More seriously :  The idea is to use 5 diskless headless "drqueue" for image processing.

Just a thought - if you're just trying to do something as simple as
run a script or program on a set of many separate files and you just
want to throw more hardware at the problem then you might be able to
get something working fairly quickly with Hudson - the Continuous
Integration server:

http://hudson-ci.org/

Although it's designed primarily for building software, you can set up
Hudson do almost anything. It supports clusters where multiple jobs
are distributed to 'slaves'. There are plugins for Hadoop too if you
want to take it further.

Just an idea. It really depends on how you can break up your job and
the bandwidth constraints you might have between the nodes and
filesystem(s). It could go from something fairly simple to something
really complicated quite quickly.

-- David.



More information about the wellylug mailing list