dschmidt opened a new pull request, #3117: URL: https://github.com/apache/tika/pull/3117
A parser builds the metadata of an embedded document beside the stream it hands over, so a length the stream already knows never reaches the metadata a client sees. The raw camera previews, the audio cover art and others come back with a type, a name and a resource path, but nothing about how many bytes they are, and a client that only wants to know how big an attachment is has to read it. The convention is established: `ZipParser` sets `Content-Length` from the zip entry, `AbstractPOIFSExtractor` from the OLE2 directory entry, `AbstractPDF2XHTML` from the embedded file specification, and RTF, PST and WARC do the same. This fills in the rest. `ParsingEmbeddedDocumentExtractor` now sets `Content-Length` where the field is blank and the stream knows its length. `hasLength()` answers from the source, so nothing is ever spooled to measure one, and a stream that would have to be read to be measured is left alone. A length the parser set stands, since it describes the item, which need not be the whole of the stream it happens to hand over. That covers every file or byte array backed embedded stream, whatever produced it. Where the length is read from the file rather than known to the stream, the parser still has to say so. `RawTiffParser` takes the preview length from the IFD, and `CoverArt` hands the picture metadata to `TikaInputStream` instead of a throwaway object, which is where the length was going before. `Content-Length` is not only an output: `TikaInputStream.get(opener, tmp, metadata)` takes the stream's length from it, so a parser that declares it also spares the embedded parse a spool. `MP4Parser`, for one, bounds its box allocation with `tis.hasLength() ? tis.getLength() : -1`. TIKA-4106 approaches the same gap through the digester, which fills the length in for callers who ask for digests, and only once the stream has been read. This is the case without digests. Tests: three in tika-core for the three properties above, and the raw preview assertions gained the length. The wider question was whether anything depends on the field being absent: tika-core (985), the standard parser package and the integration tests (3388) are green. https://issues.apache.org/jira/browse/TIKA-4873 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
