[
https://issues.apache.org/jira/browse/TIKA-4766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18131236#comment-18131236
]
ASF GitHub Bot commented on TIKA-4766:
--------------------------------------
krickert commented on code in PR #2961:
URL: https://github.com/apache/tika/pull/2961#discussion_r4355353006
##########
tika-grpc-api/README.md:
##########
@@ -38,7 +38,10 @@ metadata as a `fields` map of strings, is defined in the
`tika-grpc` module.
Tika metadata key can hold several values. The value type follows the
`Property`
type Tika declares for the key (integer, number, boolean or timestamp); keys
without a declared type are strings. For example, `pdf:charsPerPage` arrives
as
- one int64 per page. New or renamed keys need no change to the proto.
+ one int64 per page. Keys declared only in a parser module, such as
`csv:num-rows`,
+ get integer, number and boolean types only when the configuration sets
+ `"metadata-list": {"typedValues": true}`, which also types the JSON that
emitters
+ write; their timestamps stay strings. New or renamed keys need no change to
the proto.
Review Comment:
gRPC people would find this part very useful.
##########
tika-grpc/src/main/java/org/apache/tika/pipes/grpc/TikaGrpcServerImpl.java:
##########
@@ -360,65 +383,120 @@ private ParseContext
buildRequestParseContext(FetchAndParseRequest request,
return null;
}
}
- String additionalFetchConfigJson =
request.getAdditionalFetchConfigJson();
if (StringUtils.isNotBlank(additionalFetchConfigJson)) {
// The fork reads this jsonConfig by the fetcher's registered
component name
// (e.g. "http-fetcher"). An unregistered key fails the fork's
resolveAll and a
// wire-blocked one is refused by its restricted tuple
deserialization
> Replace tika-grpc fields map with a typed Document parse contract
> -----------------------------------------------------------------
>
> Key: TIKA-4766
> URL: https://issues.apache.org/jira/browse/TIKA-4766
> Project: Tika
> Issue Type: New Feature
> Components: tika-pipes
> Affects Versions: 4.0.0
> Reporter: Kristian Rickert
> Priority: Major
> Labels: grpc, pipes, protobuf
>
> Replace the flat {{FetchAndParseReply.fields}} map ({{map<string,string>}})
> with a typed parse result.
> Approach (reshaped from the original {{ParseResponse}} design after review in
> [PR #2916|https://github.com/apache/tika/pull/2916]): a single small
> {{Document}} proto (~200 lines) rather than per-format metadata messages.
> * *Content*: a structured markdown block tree -- headings, paragraphs, lists,
> tables, code blocks, inline runs (CommonMark + GFM) -- plus the rendered
> markdown string {{ToMarkdownContentHandler}} already produces (TIKA-4730).
> * *Typed common metadata*: title, authors, keywords, languages,
> created/modified as {{Timestamp}}s, page/word/character counts, dimensions,
> rights.
> * *Lossless tagged tail*: every remaining metadata key,
> multivalue-preserving, typed only where Tika's own {{Property}} declares a
> type, string otherwise -- never guessed.
> * *Embedded documents* recurse as fully typed child {{Document}}s.
> * Format specifics live in per-parser {{DocumentTransformer}} code
> ({{tika-grpc-mapper}}), never in the wire contract, so metadata churn never
> forces a client rebuild.
> Modules: {{tika-grpc-api}} (proto + generated messages + bundled
> {{FileDescriptorSet}}), {{tika-grpc-mapper}}, {{tika-grpc}} integration.
> Breaking change for clients reading {{fields}} (field number reserved).
> PR: [https://github.com/apache/tika/pull/2921] (supersedes
> [#2916|https://github.com/apache/tika/pull/2916])
> Follow-ups will be tracked in separate issues: pluggable external parsers
> (opaque {{Any}} extension results from registered gRPC services), and a
> Markdown input parser.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
