yidaimi opened a new issue, #13198: URL: https://github.com/apache/seatunnel/issues/13198
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened `MongodbSinkOptions` declares defaults for four sink options, but `MongodbSinkFactory` only passes a value to `MongodbWriterOptions.Builder` when the option is explicitly present: ```java // MongodbSinkFactory (dev, lines 78-96) if (readonlyConfig.getOptional(MongodbSinkOptions.BUFFER_FLUSH_MAX_ROWS).isPresent()) { builder.withFlushSize(readonlyConfig.get(MongodbSinkOptions.BUFFER_FLUSH_MAX_ROWS)); } // same pattern for BUFFER_FLUSH_INTERVAL, RETRY_MAX and RETRY_INTERVAL ``` When an option is omitted, the builder keeps the Java field defaults instead of the declared ones: | Option | Declared default | Effective value when omitted | |---|---|---| | `buffer-flush.max-rows` | 1000 | 0 (`protected int flushSize;`) | | `buffer-flush.interval` | 30000 ms | 0 (`protected long batchIntervalMs;`) | | `retry.max` | 3 | 0 (`protected int retryMax;`) | | `retry.interval` | 1000 ms | 0 (`protected long retryInterval;`) | Consequences: 1. With `flushSize = 0`, `MongodbWriter.isOverMaxBatchSizeLimit()` evaluates `bulkRequests.size() >= 0`, which is always true, so `doBulkWrite()` runs after **every row**: one network round trip per row instead of one per 1000 rows. 2. With `retryMax = 0`, a failed bulk write is not retried, although the documented default is 3. The first point is invisible against a local server and severe against a remote one. Measured with the 3.0.0 sink, 20,000 inserts, nothing else changed: | Target | Options omitted | `buffer-flush.max-rows = 1000` set explicitly | |---|---|---| | Amazon DocumentDB 5.0.2 (same VPC, ~3 ms RTT) | 56 s engine time | 3 s engine time | | Local MongoDB 7 (control) | 5 s engine time | - | Evidence (cases P01 / P01b / X1a) and the reproduction script: https://github.com/yidaimi/seatunnel-cosmos-to-documentdb/blob/v0.2-delete_insert/test/sink-compat/evidence-excerpts.md The issue was found while preparing the DocumentDB compatibility matrix for #10443, but it is not DocumentDB-specific: it affects every MongoDB sink job that does not set these options. **Expected behavior:** omitted options use the declared defaults (1000 rows, 30000 ms, 3 retries, 1000 ms); explicitly configured values keep working as today. **Proposed fix:** read the four options with `readonlyConfig.get(...)` unconditionally (the `Option` already carries its default), so the builder always receives either the configured or the declared value. Regression test: create the sink without any of the four options and verify the writer behaves with 1000 / 30000 / 3 / 1000, plus a control case with explicit values. ### SeaTunnel Version 3.0.0. The factory code is unchanged on current `dev` (checked at b7ef4936fe). ### SeaTunnel Config ```conf env { parallelism = 1 job.mode = "BATCH" } source { FakeSource { row.num = 20000 schema = { fields { id = bigint, name = string } } } } sink { MongoDB { uri = "mongodb://<user>:<password>@<host>:27017/?retryWrites=false" database = "test" collection = "flush_defaults" # buffer-flush.max-rows / buffer-flush.interval / retry.max / retry.interval omitted } } ``` ### Running Command ```shell ./bin/seatunnel.sh --config flush_defaults.conf -m local ``` ### Error Exception ```log No exception. The job finishes, but writes one row per bulkWrite call (see the timings above) and does not retry failed bulk writes. ``` ### Zeta or Flink or Spark Version Zeta (SeaTunnel 3.0.0) ### Java or Scala Version OpenJDK 17 (Amazon Corretto 17.0.16) ### Screenshots _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
