Sergei Lilichenko created BEAM-13162:
----------------------------------------
Summary: BigQueryIO Storage Write API method - suppress logging
API errors for known use cases.
Key: BEAM-13162
URL: https://issues.apache.org/jira/browse/BEAM-13162
Project: Beam
Issue Type: New Feature
Components: io-java-gcp
Affects Versions: 2.33.0
Reporter: Sergei Lilichenko
Many BigQuery Storage API errors get logged at ERROR level even though they are
expected and handled by the Beam's transform. They should be suppresses, or
output at DEBUG/INFO level to differentiate from abnormal behavior.
Example:
{
"jsonPayload": {
"message": "Got error io.grpc.StatusRuntimeException: ALREADY_EXISTS: The
offset is within stream, expected offset 52125, received 51264 Entity:
projects/event-processing-demo/datasets/bigquery_io/tables/events/streams/CiQ2MmZlOTFjNS0wMDAwLTIzNTItOWMxYS01ODI0MjlhOWRiOGM
closing
projects/event-processing-demo/datasets/bigquery_io/tables/events/streams/CiQ2MmZlOTFjNS0wMDAwLTIzNTItOWMxYS01ODI0MjlhOWRiOGM",
"step": "Save Rows to BigQuery/StorageApiLoads/StorageApiWriteSharded/Write
Records",
"worker": "data-processing-streaming-10250846-kps9-harness-hnj3",
"stage": "P6",
"job": "2021-10-25_08_46_20-4863073108555028756",
"thread": "258",
"work": "3800026c09938fac-5cf2f68daacdd",
"logger": "org.apache.beam.sdk.io.gcp.bigquery.StorageApiWritesShardedRecords"
},
"resource": {
"type": "dataflow_step",
"labels": {
"project_id": "event-processing-demo",
"step_id": "Save Rows to BigQuery/StorageApiLoads/StorageApiWriteSharded/Write
Records",
"job_id": "2021-10-25_08_46_20-4863073108555028756",
"region": "us-central1",
"job_name": "data-processing-streaming-storage-write-api-200-2"
}
},
"timestamp": "2021-10-25T15:57:43.617Z",
"severity": "ERROR",
"labels": {
"compute.googleapis.com/resource_type": "instance",
"dataflow.googleapis.com/job_name":
"data-processing-streaming-storage-write-api-200-2",
"dataflow.googleapis.com/log_type": "supportability",
"compute.googleapis.com/resource_id": "6365565973148954518",
"dataflow.googleapis.com/job_id": "2021-10-25_08_46_20-4863073108555028756",
"dataflow.googleapis.com/region": "us-central1",
"compute.googleapis.com/resource_name":
"data-processing-streaming-10250846-kps9-harness-hnj3"
},
"logName":
"projects/event-processing-demo/logs/dataflow.googleapis.com%2Fworker",
"receiveTimestamp": "2021-10-25T15:57:47.557344849Z"
}
Additional exceptions that fit this category:
* Got error io.grpc.StatusRuntimeException: FAILED_PRECONDITION: Stream is
closed due to com.google.api.gax.rpc.UnavailableException:
io.grpc.StatusRuntimeException: UNAVAILABLE: Connection closed after GOAWAY.
HTTP/2 error code: NO_ERROR, debug data: server_shutting_down closing
* Got error com.google.api.gax.rpc.AbortedException:
io.grpc.StatusRuntimeException: ABORTED: Closing the stream because server is
restarted. This is expected and client is advised to reconnect.
* Got error io.grpc.StatusRuntimeException: FAILED_PRECONDITION: Stream is
closed due to com.google.api.gax.rpc.AbortedException:
io.grpc.StatusRuntimeException: ABORTED: Closing the stream because server is
restarted. This is expected and client is advised to reconnect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
