You can try UdmSearch : http://search.mnogo.ru/
> Perhaps you could help me:
>
> I'm looking for a simple web search tool or spider to map all the links
> from a certain URL starting point(s) to a certain depth, and print or
> log the URLs it encounters. I've been using "wget -r -l 5" so far
check
swish-e (have problems with hebrew characters)
ht-dig (never tried)
Adi Stav wrote:
> Perhaps you could help me:
>
> I'm looking for a simple web search tool or spider to map all the links
> from a certain URL starting point(s) to a certain depth, and print or
> log the URLs it encounters.
Perhaps you could help me:
I'm looking for a simple web search tool or spider to map all the links
from a certain URL starting point(s) to a certain depth, and print or
log the URLs it encounters. I've been using "wget -r -l 5" so far but it
cannot parallelize its requests, visits the same URL
