@Anton OkolnychyiLooking forward to the pull
requests :-)
On Wed, Mar 18, 2020 at 3:38 AM Anton Okolnychyi
wrote:
> As Ryan noted, there are multiple cases when some files can be left orphan
> and we won’t be able to avoid all such cases. In our case, we are
> periodically listing locations
I am +1 on having spark-2 and spark-3 modules as well.
> On 7 Mar 2020, at 15:03, RD wrote:
>
> I'm +1 to separate modules for spark-2 and spark-3, after the 0.8 release.
> I think it would be a big change in organizations to adopt Spark-3 since that
> brings in Scala-2.12 which is binary incom
As Ryan noted, there are multiple cases when some files can be left orphan and
we won’t be able to avoid all such cases. In our case, we are periodically
listing locations and comparing the actual files to all files present in
metadata tables.
At some point, there was an agreement in the commu
