n that
> under
> > > the
> > > > > > rejected alternatives.
> > > > > >
> > > > > > Kind regards,
> > > > > > Gustavo
> > > > > >
> > > > > > Am Mo., 28. Apr. 2025 um 14:18 Uhr sc
+1 (binding)
Cheers
On Wed, May 7, 2025 at 6:37 PM Gustavo de Morais
wrote:
> Hi everyone,
>
> I'd like to start voting on FLIP-516: Multi-Way Join Operator [1]. The
> discussion can be found in this thread [2].
> The vote will be open for at least 72 hours, unless there is an objection
> or no
and use that in
> our application. If you can add a fix we can pick this up (and test this
> already 😊 ).
>
> Kind regards,
> Fred
>
> *From: *Arvid Heise
> *Date: *Tuesday, 20 May 2025 at 09:12
> *To: *Teunissen, F.G.J. (Fred)
> *Cc: *dev@flink.apache.org
> *Subject: *R
Hi Fred,
ah yes, I think I understand the issue. The KafkaSink always creates a
KafkaCommitter even if you are not using EXACTLY_ONCE. It's an unfortunate
limitation of our Sink design.
When I implemented the change, I was assuming that you are running EOS if
there is a committer (because else its
I'm also in favor of Option A of the presented options.
I had one additional idea (Option F) that mixes two approaches B+E
ROW_WISE_TABLE
SET_PARTITIONED_TABLE
You'd have ROW and SET that comes from the SQL standard but you also have a
stronger connection between SET and partitioning.
Best,
Ar
Hi David,
Flink currently has not a compelling story when it comes to error handling
and I'd like to change that.
I'd advocate for an approach that naturally translates into dead letter
queues as known from other stream processors such as KStreams. [1] With
your idea of metadata columns, you are
e only showing
> nulls in the join output. We think that in this case is could be reasonable
> to use metadata columns in the short term and show nulls in the join output?
> * Can I confirm your thinking on how we would set the schema of the
> side output [1] and for the proposed system
Hi Zexian,
the general idea and approach LGTM. A couple of questions:
* If we don't want to provide RateLimiter on operator level (seems to be
out of scope for this FLIP), can we still make the RateLimiter a property
of the Source similar to Watermark strategy and pass it through the
implementatio
Arvid Heise created FLINK-13956:
---
Summary: Add sequence file format with repeated sync blocks
Key: FLINK-13956
URL: https://issues.apache.org/jira/browse/FLINK-13956
Project: Flink
Issue Type
Arvid Heise created FLINK-13995:
---
Summary: Fix shading of the licence information of netty
Key: FLINK-13995
URL: https://issues.apache.org/jira/browse/FLINK-13995
Project: Flink
Issue Type
Arvid Heise created FLINK-14044:
---
Summary: Reducing synchronization in AsyncWaitOperator
Key: FLINK-14044
URL: https://issues.apache.org/jira/browse/FLINK-14044
Project: Flink
Issue Type
Arvid Heise created FLINK-16014:
---
Summary: S3 plugin ClassNotFoundException SAXParser
Key: FLINK-16014
URL: https://issues.apache.org/jira/browse/FLINK-16014
Project: Flink
Issue Type: Bug
Arvid Heise created FLINK-16015:
---
Summary: Refine fallback filesystems to only handle specific
filesystems
Key: FLINK-16015
URL: https://issues.apache.org/jira/browse/FLINK-16015
Project: Flink
Arvid Heise created FLINK-16219:
---
Summary: Make AsyncWaitOperator chainable again
Key: FLINK-16219
URL: https://issues.apache.org/jira/browse/FLINK-16219
Project: Flink
Issue Type: Improvement
Arvid Heise created FLINK-16587:
---
Summary: Add basic CheckpointBarrierHandler for unaligned
checkpoint
Key: FLINK-16587
URL: https://issues.apache.org/jira/browse/FLINK-16587
Project: Flink
Arvid Heise created FLINK-16972:
---
Summary: Fix non-blocking output availability logic.
Key: FLINK-16972
URL: https://issues.apache.org/jira/browse/FLINK-16972
Project: Flink
Issue Type: Bug
Arvid Heise created FLINK-17218:
---
Summary: Add recovery to integration tests of unaligned checkpoints
Key: FLINK-17218
URL: https://issues.apache.org/jira/browse/FLINK-17218
Project: Flink
Arvid Heise created FLINK-17258:
---
Summary: Enable unaligned checkpoints in tests by default
Key: FLINK-17258
URL: https://issues.apache.org/jira/browse/FLINK-17258
Project: Flink
Issue Type
Arvid Heise created FLINK-14199:
---
Summary: Only use dedicated/named classes for mailbox letters.
Key: FLINK-14199
URL: https://issues.apache.org/jira/browse/FLINK-14199
Project: Flink
Issue
Arvid Heise created FLINK-14304:
---
Summary: Avoid task starvation with mailbox
Key: FLINK-14304
URL: https://issues.apache.org/jira/browse/FLINK-14304
Project: Flink
Issue Type: Improvement
Arvid Heise created FLINK-14480:
---
Summary: Sort out exception supression during snapshotting for
non-running tasks.
Key: FLINK-14480
URL: https://issues.apache.org/jira/browse/FLINK-14480
Project
Arvid Heise created FLINK-14747:
---
Summary: Performance regression with latest changes to Mailbox
Key: FLINK-14747
URL: https://issues.apache.org/jira/browse/FLINK-14747
Project: Flink
Issue
Arvid Heise created FLINK-14794:
---
Summary: KeyedStream#transform not working with factories
Key: FLINK-14794
URL: https://issues.apache.org/jira/browse/FLINK-14794
Project: Flink
Issue Type
Arvid Heise created FLINK-14934:
---
Summary: Remove error log statement in ES connector
Key: FLINK-14934
URL: https://issues.apache.org/jira/browse/FLINK-14934
Project: Flink
Issue Type
Arvid Heise created FLINK-14935:
---
Summary: Improve mailbox exception handling
Key: FLINK-14935
URL: https://issues.apache.org/jira/browse/FLINK-14935
Project: Flink
Issue Type: Improvement
Arvid Heise created FLINK-14975:
---
Summary: Presto filesystem plugin shades wrong version of guava
Key: FLINK-14975
URL: https://issues.apache.org/jira/browse/FLINK-14975
Project: Flink
Issue
Arvid Heise created FLINK-15219:
---
Summary: LocalEnvironment is not initializing plugins
Key: FLINK-15219
URL: https://issues.apache.org/jira/browse/FLINK-15219
Project: Flink
Issue Type
Arvid Heise created FLINK-15355:
---
Summary: Nightly streaming file sink fails with unshaded hadoop
Key: FLINK-15355
URL: https://issues.apache.org/jira/browse/FLINK-15355
Project: Flink
Issue
Arvid Heise created FLINK-15583:
---
Summary: Scala walkthrough archetype does not compile on Java 11
Key: FLINK-15583
URL: https://issues.apache.org/jira/browse/FLINK-15583
Project: Flink
Issue
Arvid Heise created FLINK-15777:
---
Summary: Truncate not working with plugins (as expected)
Key: FLINK-15777
URL: https://issues.apache.org/jira/browse/FLINK-15777
Project: Flink
Issue Type
Arvid Heise created FLINK-15845:
---
Summary: Add java-based StreamingFileSink test using s3
Key: FLINK-15845
URL: https://issues.apache.org/jira/browse/FLINK-15845
Project: Flink
Issue Type
Arvid Heise created FLINK-19338:
---
Summary: New source interface cannot unregister unregistered source
Key: FLINK-19338
URL: https://issues.apache.org/jira/browse/FLINK-19338
Project: Flink
Arvid Heise created FLINK-19391:
---
Summary: Deadlock during partition update
Key: FLINK-19391
URL: https://issues.apache.org/jira/browse/FLINK-19391
Project: Flink
Issue Type: Bug
Arvid Heise created FLINK-19400:
---
Summary: Removed unused BufferPoolOwner
Key: FLINK-19400
URL: https://issues.apache.org/jira/browse/FLINK-19400
Project: Flink
Issue Type: Bug
Arvid Heise created FLINK-19441:
---
Summary: Performance regression on 24.09.2020
Key: FLINK-19441
URL: https://issues.apache.org/jira/browse/FLINK-19441
Project: Flink
Issue Type: Bug
Arvid Heise created FLINK-19442:
---
Summary: Unaligned checkpoints 1.1
Key: FLINK-19442
URL: https://issues.apache.org/jira/browse/FLINK-19442
Project: Flink
Issue Type: Bug
Reporter
Arvid Heise created FLINK-19491:
---
Summary: AvroSerializerSnapshot cannot handle large schema
Key: FLINK-19491
URL: https://issues.apache.org/jira/browse/FLINK-19491
Project: Flink
Issue Type
Arvid Heise created FLINK-19520:
---
Summary: Add reliable test randomization for checkpointing
Key: FLINK-19520
URL: https://issues.apache.org/jira/browse/FLINK-19520
Project: Flink
Issue Type
Arvid Heise created FLINK-19533:
---
Summary: Add state assignment to channel state during rescaling
Key: FLINK-19533
URL: https://issues.apache.org/jira/browse/FLINK-19533
Project: Flink
Issue
Arvid Heise created FLINK-19536:
---
Summary: Fix inconsistent priority notification
Key: FLINK-19536
URL: https://issues.apache.org/jira/browse/FLINK-19536
Project: Flink
Issue Type: Bug
Arvid Heise created FLINK-19801:
---
Summary: Add support for virtual channels
Key: FLINK-19801
URL: https://issues.apache.org/jira/browse/FLINK-19801
Project: Flink
Issue Type: Sub-task
Arvid Heise created FLINK-20030:
---
Summary: Barrier announcement causes outdated
RemoteInputChannel#numBuffersOvertaken
Key: FLINK-20030
URL: https://issues.apache.org/jira/browse/FLINK-20030
Project
Arvid Heise created FLINK-20127:
---
Summary: Benchmark unaligned checkpointing times under backpressure
Key: FLINK-20127
URL: https://issues.apache.org/jira/browse/FLINK-20127
Project: Flink
Arvid Heise created FLINK-20247:
---
Summary: Hide unused unaligned checkpoint configuration options
Key: FLINK-20247
URL: https://issues.apache.org/jira/browse/FLINK-20247
Project: Flink
Issue
Arvid Heise created FLINK-20444:
---
Summary: Chain AsyncWaitOperator to new sources
Key: FLINK-20444
URL: https://issues.apache.org/jira/browse/FLINK-20444
Project: Flink
Issue Type: Improvement
Arvid Heise created FLINK-20654:
---
Summary: Unaligned checkpoint recovery may lead to corrupted data
stream
Key: FLINK-20654
URL: https://issues.apache.org/jira/browse/FLINK-20654
Project: Flink
Arvid Heise created FLINK-21181:
---
Summary: Buffer pool is destroyed error when outputting data over
a timer after cancellation.
Key: FLINK-21181
URL: https://issues.apache.org/jira/browse/FLINK-21181
Arvid Heise created FLINK-21186:
---
Summary: RecordWriterOutput swallows interrupt state when
interrupted.
Key: FLINK-21186
URL: https://issues.apache.org/jira/browse/FLINK-21186
Project: Flink
Arvid Heise created FLINK-21346:
---
Summary: Tasks on AZP should timeout more gracefully
Key: FLINK-21346
URL: https://issues.apache.org/jira/browse/FLINK-21346
Project: Flink
Issue Type
Arvid Heise created FLINK-21378:
---
Summary: Rescale pointwise connection during unaligned checkpoint
recovery
Key: FLINK-21378
URL: https://issues.apache.org/jira/browse/FLINK-21378
Project: Flink
Arvid Heise created FLINK-21452:
---
Summary: FLIP-27 sources cannot reliably downscale
Key: FLINK-21452
URL: https://issues.apache.org/jira/browse/FLINK-21452
Project: Flink
Issue Type
Arvid Heise created FLINK-21797:
---
Summary: Performance regression on 03/11/21
Key: FLINK-21797
URL: https://issues.apache.org/jira/browse/FLINK-21797
Project: Flink
Issue Type: Improvement
Arvid Heise created FLINK-21936:
---
Summary: Disable checkpointing of inflight data in pointwise
connections for unaligned checkpoints
Key: FLINK-21936
URL: https://issues.apache.org/jira/browse/FLINK-21936
Arvid Heise created FLINK-21945:
---
Summary: Disable checkpointing of inflight data in pointwise
connections for unaligned checkpoints
Key: FLINK-21945
URL: https://issues.apache.org/jira/browse/FLINK-21945
Arvid Heise created FLINK-21992:
---
Summary: Investigate potential buffer leak in unaligned checkpoint
Key: FLINK-21992
URL: https://issues.apache.org/jira/browse/FLINK-21992
Project: Flink
Arvid Heise created FLINK-22053:
---
Summary: Recovery of a completed split in NumberSequenceSource
causes fatal exception.
Key: FLINK-22053
URL: https://issues.apache.org/jira/browse/FLINK-22053
Project
Arvid Heise created FLINK-22136:
---
Summary: Device application for unaligned checkpoint test on
cluster
Key: FLINK-22136
URL: https://issues.apache.org/jira/browse/FLINK-22136
Project: Flink
Arvid Heise created FLINK-22137:
---
Summary: Execute unaligned checkpoint test on a cluster
Key: FLINK-22137
URL: https://issues.apache.org/jira/browse/FLINK-22137
Project: Flink
Issue Type: Sub
Arvid Heise created FLINK-22290:
---
Summary: Add new unaligned checkpiont options to Python API.
Key: FLINK-22290
URL: https://issues.apache.org/jira/browse/FLINK-22290
Project: Flink
Issue Type
Arvid Heise created FLINK-22573:
---
Summary: AsyncIO can timeout elements after completion
Key: FLINK-22573
URL: https://issues.apache.org/jira/browse/FLINK-22573
Project: Flink
Issue Type: Bug
Arvid Heise created FLINK-22682:
---
Summary: Checkpoint interval too large for higher DOP
Key: FLINK-22682
URL: https://issues.apache.org/jira/browse/FLINK-22682
Project: Flink
Issue Type: Bug
Arvid Heise created FLINK-22686:
---
Summary: Incompatible subtask mappings while resuming from
unaligned checkpoints
Key: FLINK-22686
URL: https://issues.apache.org/jira/browse/FLINK-22686
Project: Flink
Arvid Heise created FLINK-22828:
---
Summary: Allow using a custom AWS credentials provider for the
Kinesis Connector
Key: FLINK-22828
URL: https://issues.apache.org/jira/browse/FLINK-22828
Project: Flink
Arvid Heise created FLINK-22876:
---
Summary: Adding SharedObjects junit rule to ease test development
Key: FLINK-22876
URL: https://issues.apache.org/jira/browse/FLINK-22876
Project: Flink
Issue
Arvid Heise created FLINK-22902:
---
Summary: Port KafkaSink to FLIP-143
Key: FLINK-22902
URL: https://issues.apache.org/jira/browse/FLINK-22902
Project: Flink
Issue Type: Improvement
Arvid Heise created FLINK-22964:
---
Summary: Connector-base exposes dependency to flink-core.
Key: FLINK-22964
URL: https://issues.apache.org/jira/browse/FLINK-22964
Project: Flink
Issue Type
Arvid Heise created FLINK-23024:
---
Summary: RPC result TaskManagerInfoWithSlots not serializable
Key: FLINK-23024
URL: https://issues.apache.org/jira/browse/FLINK-23024
Project: Flink
Issue
Arvid Heise created FLINK-23617:
---
Summary: Co-locate sink operators with same parallelism
Key: FLINK-23617
URL: https://issues.apache.org/jira/browse/FLINK-23617
Project: Flink
Issue Type
Arvid Heise created FLINK-23621:
---
Summary: FLIP-177: Extend Sink API
Key: FLINK-23621
URL: https://issues.apache.org/jira/browse/FLINK-23621
Project: Flink
Issue Type: Improvement
Arvid Heise created FLINK-23652:
---
Summary: Implement FLIP-179
Key: FLINK-23652
URL: https://issues.apache.org/jira/browse/FLINK-23652
Project: Flink
Issue Type: Improvement
Arvid Heise created FLINK-23722:
---
Summary: S3 Tests fail on AZP: Unable to find a region via the
region provider chain. Must provide an explicit region in the builder or setup
environment to supply a region.
Key: FLINK-23722
Arvid Heise created FLINK-23759:
---
Summary: notifyCheckpointComplete without corresponding
snapshotState
Key: FLINK-23759
URL: https://issues.apache.org/jira/browse/FLINK-23759
Project: Flink
Arvid Heise created FLINK-23767:
---
Summary: FLIP-180: Adjust StreamStatus and Idleness definition
Key: FLINK-23767
URL: https://issues.apache.org/jira/browse/FLINK-23767
Project: Flink
Issue
Arvid Heise created FLINK-23803:
---
Summary: Improve instantiation of InMemoryReporter
Key: FLINK-23803
URL: https://issues.apache.org/jira/browse/FLINK-23803
Project: Flink
Issue Type
Arvid Heise created FLINK-23807:
---
Summary: Use metrics to detect restarts in
MiniClusterTestEnvironment#triggerTaskManagerFailover
Key: FLINK-23807
URL: https://issues.apache.org/jira/browse/FLINK-23807
Arvid Heise created FLINK-23816:
---
Summary: Test new (Kafka) metrics in cluster
Key: FLINK-23816
URL: https://issues.apache.org/jira/browse/FLINK-23816
Project: Flink
Issue Type: Sub-task
Arvid Heise created FLINK-23817:
---
Summary: Write documentation for standardized operator metrics
Key: FLINK-23817
URL: https://issues.apache.org/jira/browse/FLINK-23817
Project: Flink
Issue
Arvid Heise created FLINK-23858:
---
Summary: Clarify StreamRecord#timestamp.
Key: FLINK-23858
URL: https://issues.apache.org/jira/browse/FLINK-23858
Project: Flink
Issue Type: Technical Debt
Arvid Heise created FLINK-23969:
---
Summary: Test Pulsar source end 2 end
Key: FLINK-23969
URL: https://issues.apache.org/jira/browse/FLINK-23969
Project: Flink
Issue Type: Sub-task
Arvid Heise created FLINK-24108:
---
Summary: Translate KafkaSink documentation to Chinese
Key: FLINK-24108
URL: https://issues.apache.org/jira/browse/FLINK-24108
Project: Flink
Issue Type
Arvid Heise created FLINK-24182:
---
Summary: Tasks canceler should not immediately interrupt
Key: FLINK-24182
URL: https://issues.apache.org/jira/browse/FLINK-24182
Project: Flink
Issue Type
Arvid Heise created FLINK-24250:
---
Summary: Add De/Serialization API to tear-down user code
Key: FLINK-24250
URL: https://issues.apache.org/jira/browse/FLINK-24250
Project: Flink
Issue Type
Arvid Heise created FLINK-24252:
---
Summary: HybridSource should return TypeInformation
Key: FLINK-24252
URL: https://issues.apache.org/jira/browse/FLINK-24252
Project: Flink
Issue Type
Arvid Heise created FLINK-24574:
---
Summary: Allow metrics to be removed in UDFs
Key: FLINK-24574
URL: https://issues.apache.org/jira/browse/FLINK-24574
Project: Flink
Issue Type: Improvement
Arvid Heise created FLINK-17580:
---
Summary: NPE in unaligned checkpoint for EndOfPartition events
Key: FLINK-17580
URL: https://issues.apache.org/jira/browse/FLINK-17580
Project: Flink
Issue
Arvid Heise created FLINK-17640:
---
Summary:
RecoveredInputChannelTest.testConcurrentReadStateAndProcessAndRelease() failed
Key: FLINK-17640
URL: https://issues.apache.org/jira/browse/FLINK-17640
Project
Arvid Heise created FLINK-17643:
---
Summary: LaunchCoordinatorTest fails
Key: FLINK-17643
URL: https://issues.apache.org/jira/browse/FLINK-17643
Project: Flink
Issue Type: Bug
Arvid Heise created FLINK-17780:
---
Summary: Add task name to log statements of ChannelStateWriter
Key: FLINK-17780
URL: https://issues.apache.org/jira/browse/FLINK-17780
Project: Flink
Issue
Arvid Heise created FLINK-17819:
---
Summary: Yarn error unhelpful when forgetting HADOOP_CLASSPATH
Key: FLINK-17819
URL: https://issues.apache.org/jira/browse/FLINK-17819
Project: Flink
Issue
Arvid Heise created FLINK-18094:
---
Summary: Index offset not consistent in CheckpointedInputGate for
unaligned checkpoints.
Key: FLINK-18094
URL: https://issues.apache.org/jira/browse/FLINK-18094
Arvid Heise created FLINK-18139:
---
Summary: Unaligned checkpoints checks wrong channels for inflight
data.
Key: FLINK-18139
URL: https://issues.apache.org/jira/browse/FLINK-18139
Project: Flink
Arvid Heise created FLINK-18252:
---
Summary: Resuming Savepoint (file, async, no parallelism change)
end-to-end test fails with unaligned checkpoint enabled
Key: FLINK-18252
URL: https://issues.apache.org/jira/browse
Arvid Heise created FLINK-18403:
---
Summary: Ensure that only exactly once checkpointing can be
unaligned
Key: FLINK-18403
URL: https://issues.apache.org/jira/browse/FLINK-18403
Project: Flink
Arvid Heise created FLINK-18404:
---
Summary: Make unaligned checkpoints rescalable
Key: FLINK-18404
URL: https://issues.apache.org/jira/browse/FLINK-18404
Project: Flink
Issue Type: Improvement
Arvid Heise created FLINK-18405:
---
Summary: Add watermark support for unaligned checkpoints
Key: FLINK-18405
URL: https://issues.apache.org/jira/browse/FLINK-18405
Project: Flink
Issue Type
Arvid Heise created FLINK-18773:
---
Summary: Enable parallel classloading
Key: FLINK-18773
URL: https://issues.apache.org/jira/browse/FLINK-18773
Project: Flink
Issue Type: Improvement
Arvid Heise created FLINK-18851:
---
Summary: Add checkpoint type to checkpoint history entries in Web
UI
Key: FLINK-18851
URL: https://issues.apache.org/jira/browse/FLINK-18851
Project: Flink
Arvid Heise created FLINK-19026:
---
Summary: Improve threading model of Unaligner
Key: FLINK-19026
URL: https://issues.apache.org/jira/browse/FLINK-19026
Project: Flink
Issue Type: Improvement
Arvid Heise created FLINK-8715:
--
Summary: RocksDB does propagate reconfiguration of serializer to
the states
Key: FLINK-8715
URL: https://issues.apache.org/jira/browse/FLINK-8715
Project: Flink
Arvid Heise created FLINK-8716:
--
Summary: AvroSerializer does not use schema of snapshot
Key: FLINK-8716
URL: https://issues.apache.org/jira/browse/FLINK-8716
Project: Flink
Issue Type: Bug
301 - 400 of 429 matches
Mail list logo
