[
https://issues.apache.org/jira/browse/BEAM-9674?focusedWorklogId=423044&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-423044
]
ASF GitHub Bot logged work on BEAM-9674:
----------------------------------------
Author: ASF GitHub Bot
Created on: 15/Apr/20 20:38
Start Date: 15/Apr/20 20:38
Worklog Time Spent: 10m
Work Description: kmjung commented on pull request #11292: [BEAM-9674]
Don't specify selected fields when fetching BigQuery table size
URL: https://github.com/apache/beam/pull/11292#discussion_r409118981
##########
File path:
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIOStorageReadTest.java
##########
@@ -1368,19 +1368,14 @@ public void
testStreamSourceSplitAtFractionFailsWhenParentIsPastSplitPoint() thr
public void testReadFromBigQueryIO() throws Exception {
fakeDatasetService.createDataset("foo.com:project", "dataset", "", "",
null);
TableReference tableRef =
BigQueryHelpers.parseTableSpec("foo.com:project:dataset.table");
-
- Table table =
- new Table().setTableReference(tableRef).setNumBytes(10L).setSchema(new
TableSchema());
-
+ Table table = new
Table().setTableReference(tableRef).setNumBytes(10L).setSchema(TABLE_SCHEMA);
fakeDatasetService.createTable(table);
CreateReadSessionRequest expectedCreateReadSessionRequest =
CreateReadSessionRequest.newBuilder()
.setParent("projects/project-id")
.setTableReference(BigQueryHelpers.toTableRefProto(tableRef))
.setRequestedStreams(10)
- .setReadOptions(
Review comment:
I'm not sure that I understand your question here. We used to have e2e tests
for BigQueryIO with the read API only for the case where the caller specified
all fields as selected fields; with this change, now we have two tests -- one
which covers the case where no selected fields are covered (which is
effectively the same as the case where all fields are specified), and another
which covers the case where only a subset of fields are specified.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 423044)
> "Selected fields list too long" error when calling tables.get in
> BigQueryStorageTableSource
> -------------------------------------------------------------------------------------------
>
> Key: BEAM-9674
> URL: https://issues.apache.org/jira/browse/BEAM-9674
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Affects Versions: 2.19.0
> Reporter: Kenneth Jung
> Assignee: Kenneth Jung
> Priority: Minor
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> Customers experience errors similar to the following:
> Caused by:
> com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad
> Request { "code" : 400, "errors" : [
> { "domain" : "global", "message" : "Selected fields too long: <fields> must
> be less than 16384 characters.", "reason" : "invalid" }
> ], "message" : "Selected fields too long: <fields> must be less than 16384
> characters.", "status" : "INVALID_ARGUMENT" }
> com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
>
> com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
>
> com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
>
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
> com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1097)
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
>
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
>
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
>
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl.executeWithRetries(BigQueryServicesImpl.java:938)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
