vinothchandar commented on issue #1764:
URL: https://github.com/apache/hudi/issues/1764#issuecomment-650366410
>I am just thinking if we really need to wait for all files to appear here,
or even if we need to wait, if at the end of the wait period the file is not
present it should be safe to assume that file never got created.
the marker file being like a write ahead log for parquet file is probably a
better design to keep.. In the case, that happy path of checking after 2000ms
```
// time between successive attempts to ensure written data's metadata is
consistent on storage
private static final String INITIAL_CONSISTENCY_CHECK_INTERVAL_MS_PROP =
"hoodie.consistency.check.initial_interval_ms";
private static long DEFAULT_INITIAL_CONSISTENCY_CHECK_INTERVAL_MS = 2000L;
```
should succeed most of the time right - in just one attempt right? are you
suggesting we don't even do this check to overcome the throttling issue
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
