tandonraghav edited a comment on issue #2151:
URL: https://github.com/apache/hudi/issues/2151#issuecomment-705665216
@bvaradar Mongo Collection-> Hudi Table (This is our setup also)
- We cannot put collections to different topics because there are 100s of
such collections, so not in favour of creating those many topics.
- I am using Spark DS to write Records and do compaction in a different
spark job as below. Do you see any issue in this approach?
````
Option<String>
instantTime=client.scheduleCompaction(Option.empty());
client.compact(instantTime.get());
````
But if this code fails somehow then how do i get the last compaction
instantTime? So, that I can use to compact again.
Regarding DeltaStreamer, I guess it works with only one table.
MultiDeltaStreamer I have not experimented with.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
