[
https://issues.apache.org/jira/browse/NIFI-14495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17947142#comment-17947142
]
Pierre Villard commented on NIFI-14495:
---------------------------------------
OK, so I looked into it a bit more...
We are currently using the ProtoRows approach instead of JsonRows approach in
the Storage Write API. Proto requires the conversion of the timestamp into a
long to construct the message so that's why the timestamp format is required in
the JSON Record Reader. If we had picked JSON, then we could have sent the
field as a String and let BigQuery do the conversion. That is not an option
right now.
Since you have a mixed of different patterns, that's annoying... What I can
think of is for you to use a processor before PutBigQuery to fix the data and
have single pattern to use in the reader (ConvertRecord for example)... or
split the data into two streams (QueryRecord on the field timestamp if it
contains Z for example) and have two PutBigQuery with two JSON readers and have
different patterns for the timestamp.
> The PutBigQuery processor fails to handle timestamp fields provided in string
> format
> ------------------------------------------------------------------------------------
>
> Key: NIFI-14495
> URL: https://issues.apache.org/jira/browse/NIFI-14495
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 2.3.0
> Reporter: Andriy
> Priority: Major
> Attachments: image-2025-04-24-13-42-00-184.png,
> image-2025-04-24-13-43-14-598.png, image-2025-04-24-13-48-09-144.png,
> image-2025-04-24-14-11-06-239.png
>
>
> The PutBigQuery processor fails to handle timestamp fields provided in string
> format. This functionality worked correctly in the processors
> PutBigQueryBatch and PutBigQueryStream.
> *Error Message:*
> {{Wrong object type used with protocol message reflection. Field number: 11,
> field java type: LONG, value type: java.lang.String}}
> *Example of Input:*
> {
> "timestamp": "2025-04-03T03:44:01.234Z"
> },
> {
> "timestamp": "2025-04-11 10:07:31.072000"
> }
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
