[wellylug] spider needed

[jumbophut]

On Tue, 31 Aug 2004 11:34:11 +1200, Brenda O'Hagan wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> i need to spider a webstie for an automatic mirroring script.
> the site is in wml
> 

Sounds like the sort of thing rsync would be ideal for, although the
owner of the site to be mirrored would have to open up the file system
to you (you can run rsync over ssh to be secure).  It's not a spider,
so it doesn't follow links in documents, but it is designed to mirror
whole directories recursively, given direct access to the file system.
 rsync has to be installed at both ends of the connection.

-- 
Tony (echo 'spend!,pocket awide' | sed 'y/acdeikospntw!, /l at omcgtjuba.phi/')