[
https://issues.apache.org/jira/browse/BEAM-14004?focusedWorklogId=773510&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-773510
]
ASF GitHub Bot logged work on BEAM-14004:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 23/May/22 14:35
Start Date: 23/May/22 14:35
Worklog Time Spent: 10m
Work Description: johnjcasey commented on code in PR #17511:
URL: https://github.com/apache/beam/pull/17511#discussion_r879531966
##########
sdks/java/io/mongodb/src/main/java/org/apache/beam/sdk/io/mongodb/MongoDbIO.java:
##########
@@ -489,8 +490,16 @@ public List<BoundedSource<Document>> split(
// maxChunkSize is the Mongo partition size in MB
LOG.debug("Splitting in chunk of {} MB", desiredBundleSizeBytes /
1024 / 1024);
splitVectorCommand.append("maxChunkSize", desiredBundleSizeBytes /
1024 / 1024);
- Document splitVectorCommandResult =
mongoDatabase.runCommand(splitVectorCommand);
- splitKeys = (List<Document>)
splitVectorCommandResult.get("splitKeys");
+ try {
Review Comment:
In principle, this should be written such that we can mock out the database,
so we can throw an exception when we want. It looks like the existing tests use
an embedded instance of mongo. Is there any way we can tell that instance to
throw an exception?
Issue Time Tracking
-------------------
Worklog Id: (was: 773510)
Time Spent: 9h 40m (was: 9.5h)
> [CosmosDB] com.mongodb.MongoCommandException: Command failed with error 115:
> Command is not supported
> -----------------------------------------------------------------------------------------------------
>
> Key: BEAM-14004
> URL: https://issues.apache.org/jira/browse/BEAM-14004
> Project: Beam
> Issue Type: Bug
> Components: io-java-mongodb
> Affects Versions: 2.36.0
> Reporter: Hector Miuler Malpica Gallegos
> Priority: P2
> Fix For: 2.36.0
>
> Time Spent: 9h 40m
> Remaining Estimate: 0h
>
> You can't read cosmosdb collections using mongodb api because mongodb api
> doesn't support the *{{splitVector}}* command.
>
>
> {code:java}
> Sending command '{"splitVector": "test01.payment", "keyPattern": {"_id": 1},
> "force": false, "maxChunkSize": 1}' with request id 22 to database test01 on
> connection [connectionId{localValue:6, serverValue:792577514}] to server
> cdb-test.documents.azure.com:10255{code}
>
> {code:java}
> com.mongodb.MongoCommandException: Command failed with error 115: 'Command is
> not supported. ConnectionId fb598622-cc64-439a-affe-188c280c8e52
> ActivityId: fde0dba1-0000-0000-0000-000000000000' on server
> cdb-test.documents.azure.com:10255. The full response is{"_t":
> "OKMongoResponse", "ok": 0, "code": 115, "errmsg": "Command is not
> supported. ConnectionId fb598622-cc64-439a-affe-188c280c8e52 ActivityId:
> fde0dba1-0000-0000-0000-000000000000", "$err": "Command is not supported.
> ConnectionId fb598622-cc64-439a-affe-188c280c8e52 ActivityId:
> fde0dba1-0000-0000-0000-000000000000"} at
> com.mongodb.internal.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:175)
> at
> com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:303)
> at
> com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:259)
> at
> com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:99)
> at
> com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:450)
> at
> com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:72)
> at
> com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:226)
> at
> com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:269)
> at
> com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:131)
> at
> com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:123)
> at
> com.mongodb.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:343)
> at
> com.mongodb.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:334)
> at
> com.mongodb.operation.CommandOperationHelper.executeCommandWithConnection(CommandOperationHelper.java:220)
> at
> com.mongodb.operation.CommandOperationHelper$5.call(CommandOperationHelper.java:206)
> at
> com.mongodb.operation.OperationHelper.withReadConnectionSource(OperationHelper.java:463)
> at
> com.mongodb.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:203)
> at
> com.mongodb.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:198)
> at
> com.mongodb.operation.CommandReadOperation.execute(CommandReadOperation.java:59)
> at
> com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:194)
> at
> com.mongodb.client.internal.MongoDatabaseImpl.executeCommand(MongoDatabaseImpl.java:194)
> at
> com.mongodb.client.internal.MongoDatabaseImpl.runCommand(MongoDatabaseImpl.java:163)
> at
> com.mongodb.client.internal.MongoDatabaseImpl.runCommand(MongoDatabaseImpl.java:158)
> at
> com.mongodb.client.internal.MongoDatabaseImpl.runCommand(MongoDatabaseImpl.java:148)
> at
> org.apache.beam.sdk.io.mongodb.MongoDbIO$BoundedMongoDbSource.split(MongoDbIO.java:492)
> at
> org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$InputProvider.getInitialInputs(BoundedReadEvaluatorFactory.java:216)
> at
> org.apache.beam.runners.direct.ReadEvaluatorFactory$InputProvider.getInitialInputs(ReadEvaluatorFactory.java:88)
> at
> org.apache.beam.runners.direct.RootProviderRegistry.getInitialInputs(RootProviderRegistry.java:80)
> at
> org.apache.beam.runners.direct.ExecutorServiceParallelExecutor.start(ExecutorServiceParallelExecutor.java:160)
> at
> org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:213)
> at
> org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:67)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:323)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:309)
> ...{code}
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
