On 2013-10-01, Damjan Jovanovic wrote: > On Tue, Oct 1, 2013 at 6:09 AM, Stefan Bodewig <bode...@apache.org> wrote:
>> Reading may be simpler, here you can store the meta-information from the >> start of the file in memory and then read entries as you go, ZipFile >> inside the zip package does something like this. > From what I remember: > The "meta-information" can be anywhere in the file, as can the > compressed files themselves. The 7zip tool seems to write the > meta-information at the end of the 7z file when multi-file archives > are created. Oh yes, my understanding has been pretty much wrong and re-reading your implementation has helped me to see clearer. Right now I think the important metadata actually is at the end but there is a smaller part at the front - in particular a pointer to the Header holding the metadata. > Compressed file codecs, positions, lengths, and solid compression > details are only stored in the meta-information, so it's not possible > to write a streaming reader without O(n) memory in the worst case. I agree. > Writing also requires seeking or O(n) memory, as the initial header at > the beginning of the file contains the offset to the next header, and > we only know the size/contents/location of the next header once all > the files have been written. or a temporary file to which the first header could be prepended - but if you have that, you could use seeking as well. So yes, I agree again. > Since we now have multiple archivers that require seeking, I suggest > we add a SeekableStream class or something along those lines. The > Commons Imaging project also has the same problem to solve for images, > and it uses ByteSources, which can be arrays, files, or an InputStream > wrapper that caches what has been read (so seeking is efficient, while > it only reads as much from the InputStream as is necessary). Interesting idea. Right now I'm willing to postpone and streaming API for 7z and rather cut a release with a files only API. Stefan --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
