As far as I understand there will be no thread pools coming into commons-compress, this has to be done by the caller. The caller will basically in some shape or another keep at ThreadLocal< ZipArchiveOutputStream> that will receive entries for each thread. When everyone is finished they will be merged (or maybe just appended to the "one" ZipArchiveOutputStream that is "master" and already pointing to the final file).
All c-compress needs to do is have support for transferring already compressed content from one outputstream to another, probably by reversing the outputstream back to an inputstream and appending all the entries to a different outputstream through regular iteration. There is also the issue of determining if the MANIFEST.MF file really needs to be the first file in the jar, which puts an interesting constraint on the parallelism. I have been unable to understand if the jar spec actually requires this or if it's just stuff from the old days. In maven we use plexus-archiver, which allows higher-level creation of zip archives (add a bunch of "source" specifications to archiver and say "execute" and you get your zip/tar files etc). The executor will be in plexus-archiver for our code. I've been tweaking this code for some time now with an eye to parallelize the whole thing; som now I'm moving for the kill :) Kristian 2014-12-15 13:06 GMT+01:00 Gary Gregory <garydgreg...@gmail.com>: > Sounds cool. Do you plan on allowing an executor service to be passed in to > some APIs? > > Gary > > <div>-------- Original message --------</div><div>From: Kristian Rosenvold > <krosenv...@apache.org> </div><div>Date:12/15/2014 02:49 (GMT-05:00) > </div><div>To: Commons Developers List <dev@commons.apache.org> > </div><div>Cc: </div><div>Subject: [compress] Changes to allow multithreaded > creation of zip files </div><div> > </div>I just thought I'd let you know I'm working on changes to allow > multiple threads to output to different ZipArchiveOutputStream > instances (backed by commons-io DeferredFileOutputStream or similar) > and then stitch the results back together as a single Zip file. I'm > currently just researching the scope of the required changes (which > seems to be fairly small) and working on test cases. I expect to be > working with this for a few weeks, my code can be seen at > https://github.com/krosenvold/commons-compress > > (And I'll be implementing this in maven once it's good). > > Kristian > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
