Hi Péter, thanks for bringing this up.
I don't think using a tag to "lock" a table is a good idea. The doc calls
out that this is necessary "Since Flink doesn’t provide an out of the box
solution for downstream operators sending feedback to upstream operators"
so this feels like using Iceberg met
Just curious—why did we originally introduce %1F as a separator?
When we were first discussing how to send multi-part namespace identifiers,
there was a choice about how to encode them. I advocated for using . as we
do for column names, but there were people that felt strongly about needing
to use
With 13 +1 and no -1 or +0 votes, this passes. I'll merge the PR. Thanks,
everyone!
On Thu, Aug 1, 2024 at 2:52 PM Steve Zhang
wrote:
> +1 (non-binding)
>
> Thanks,
> Steve Zhang
>
>
>
> On Aug 1, 2024, at 2:25 PM, John Zhuge wrote:
>
> +1 (non-binding)
>
> On Thu, Aug 1, 2024 at 10:55 AM Amogh
Thank you for the feedback. I've put some replies inline to specific
points below.
> "Request Changes" should only be used to literally block a PR from being
> merged - either for timing issues (e.g. "can only be merged right after
> a release is out, because it changes public docs") or because
I also don't feel it is the best fit to use tags to implement locks for
passing control messages. This is the main sticking point for me from the
design doc. However, we haven't been able to come up with a better solution
yet. Maybe we need to go back to the drawing board again.
I am also not sure
Thanks Ryan for reaching out. It's great to have a path ahead for everyone.
Thanks,
Manu
On Fri, Aug 2, 2024 at 11:55 PM Ryan Blue
wrote:
> To follow up on this, I also reached out to Manu who was the only -1 vote.
> I can understand his concern about forcing people to stay on the 1.6
> release
Not familiar with Flink, I'm wondering how Flink resolves concurrency
issues in common Flink use cases. For example, how does Flink prevent two
jobs from writing to the same file?
On the other hand, an Iceberg tag is eventually an atomic change to a file.
It's the same as using a file lock. I don'
Thanks everyone for your answers! I really appreciate it, especially since
these come into during the weekend, using your own time.
@Manu, during our initial discussion, you have mentioned that you had
similar issues with Spark compactions. You needed locking there. Is it
still an issue?
If Spark
