[
https://issues.apache.org/jira/browse/TIKA-4873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18110789#comment-18110789
]
ASF GitHub Bot commented on TIKA-4873:
--------------------------------------
dschmidt opened a new pull request, #3117:
URL: https://github.com/apache/tika/pull/3117
A parser builds the metadata of an embedded document beside the stream it
hands over, so a length the stream already knows never reaches the metadata a
client sees. The raw camera previews, the audio cover art and others come back
with a type, a name and a resource path, but nothing about how many bytes they
are, and a client that only wants to know how big an attachment is has to read
it.
The convention is established: `ZipParser` sets `Content-Length` from the
zip entry, `AbstractPOIFSExtractor` from the OLE2 directory entry,
`AbstractPDF2XHTML` from the embedded file specification, and RTF, PST and WARC
do the same. This fills in the rest.
`ParsingEmbeddedDocumentExtractor` now sets `Content-Length` where the field
is blank and the stream knows its length. `hasLength()` answers from the
source, so nothing is ever spooled to measure one, and a stream that would have
to be read to be measured is left alone. A length the parser set stands, since
it describes the item, which need not be the whole of the stream it happens to
hand over. That covers every file or byte array backed embedded stream,
whatever produced it.
Where the length is read from the file rather than known to the stream, the
parser still has to say so. `RawTiffParser` takes the preview length from the
IFD, and `CoverArt` hands the picture metadata to `TikaInputStream` instead of
a throwaway object, which is where the length was going before.
`Content-Length` is not only an output: `TikaInputStream.get(opener, tmp,
metadata)` takes the stream's length from it, so a parser that declares it also
spares the embedded parse a spool. `MP4Parser`, for one, bounds its box
allocation with `tis.hasLength() ? tis.getLength() : -1`.
TIKA-4106 approaches the same gap through the digester, which fills the
length in for callers who ask for digests, and only once the stream has been
read. This is the case without digests.
Tests: three in tika-core for the three properties above, and the raw
preview assertions gained the length. The wider question was whether anything
depends on the field being absent: tika-core (985), the standard parser package
and the integration tests (3388) are green.
https://issues.apache.org/jira/browse/TIKA-4873
> Set Content-Length on embedded documents where the parser already knows it
> ---------------------------------------------------------------------------
>
> Key: TIKA-4873
> URL: https://issues.apache.org/jira/browse/TIKA-4873
> Project: Tika
> Issue Type: Improvement
> Reporter: Dominik Schmidt
> Priority: Major
>
> Several parsers emit embedded documents without a size. A client gets the
> type,
> the name and the resource path, but has to read the bytes to learn how many
> there are. Examples: the raw camera previews from RawTiffParser, the audio
> cover
> art from CoverArt, and the motion photo video from TIKA-4869 until it was
> added
> there.
> The convention is established elsewhere: ZipParser sets Content-Length from
> the
> zip entry (along with Zip:UncompressedSize), AbstractPOIFSExtractor from the
> OLE2 directory entry, AbstractPDF2XHTML from the embedded file specification,
> and RTF, PST and WARC do the same. In each of these the parser has the size in
> hand before it hands the stream over, and in the cases listed above it does
> too:
> the JPEG preview length comes from the IFD, the cover art length from the ID3
> or
> MP4 frame.
> Content-Length is not only an output. TikaInputStream.get(opener, tmp,
> metadata)
> takes the stream's length from it, so setting it before the embedded parse
> means
> that parse knows the length without spooling to measure it: MP4Parser, for
> one,
> bounds its box allocation with tis.hasLength() ? tis.getLength() : -1.
> TIKA-4106 approaches the same gap from the other end, by having the digester
> fill the length in. That covers callers who want digests, and only once the
> stream has been read; it does not help a caller who only wants to know how big
> an attachment is.
> Two steps, either of them useful on its own:
> 1. Set it in the parsers that already know the size, starting with
> RawTiffParser
> and CoverArt.
> 2. In ParsingEmbeddedDocumentExtractor.parseEmbedded, fill Content-Length when
> it is blank and tis.hasLength() is true. That costs nothing, never spools,
> and covers every embedded stream that is file or byte array backed,
> whatever
> parser produced it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
