Hi Nurullah,
Nurullah Akkaya writes:
> Yes but AFAIK you only get availableProcessors + 2 threads with pmap
That's good to know, is this documented somewhere?
>
> which is fine when the task is CPU bound but for downloading web pages
> most of the time will be lost at waiting for I/O so havin
On May 13, 2010, at 7:52 AM, Nurullah Akkaya wrote:
> Yes but AFAIK you only get availableProcessors + 2 threads with pmap
> which is fine when the task is CPU bound but for downloading web pages
> most of the time will be lost at waiting for I/O so having more
> threads would speed things up.
Se
Yes but AFAIK you only get availableProcessors + 2 threads with pmap
which is fine when the task is CPU bound but for downloading web pages
most of the time will be lost at waiting for I/O so having more
threads would speed things up.
Regards...
--
Nurullah Akkaya
http://nakkaya.com
On Thu, May
Wouldn't this be simpler with pmap, e.g. http://gist.github.com/399269
although to be honest I don't really know how the automatically
parallelized clojure functions decide how many threads to use. Is the
JVM smart enough to only create as many system-level threads as make
sense on my hardware?
Since you don't need coordination or keep some sort of state, IMHO
future is better suited for this.
Following gist is my take, it first reads the file that contains the
list of URLs to be downloaded then splits the list into number of
thread pieces. Each future object gets a piece of the list and
Hallo all,
A friend of mine ask if there is a smart way to get the html code of
couple thousand links but with a script, it takes for ever since it
always has takes a couple of seconds to get the connection.
I needs to be multi threaded so we can use all of the download rate.
So I sad I could tr
