Jingsong Lee created FLINK-17686:
Summary: Add document to dataGen, print, blackhole connectors
Key: FLINK-17686
URL: https://issues.apache.org/jira/browse/FLINK-17686
Project: Flink
Issue Ty
I agree with the pragmatic solution.
Concerning the stability guarantees I will start a separate discussion
thread whether to provide stricter guarantees or not.
Cheers,
Till
On Wed, May 13, 2020 at 10:13 PM Thomas Weise wrote:
> On Wed, May 13, 2020 at 1:02 PM Piotr Nowojski
> wrote:
>
> > H
Dear community,
in the latest 1.10.1 bug fix release I introduced a binary incompatible
change to a class which is annotated with @PublicEvolving [1]. While this
change is technically ok since we only provide API and binary compatibility
for @Public classes across releases, it raised the question
The discuss thread for tightening the stability guarantees for
@PublicEvolving classes can be found here [1]
[1]
https://lists.apache.org/thread.html/rb0d0f887b291a490ed3773352c90ddf5f11e3d882dc501e3b8cf0ed0%40%3Cdev.flink.apache.org%3E
Cheers,
Till
On Thu, May 14, 2020 at 10:13 AM Till Rohrmann
Ack'ed
Thanks for updating the release guide Chesnay.
Cheers,
Till
On Wed, May 13, 2020 at 3:42 PM Chesnay Schepler wrote:
> Hello,
>
> I have just amended the release guide
> <
> https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release>
>
> to specify that older releases sh
Thanks for the update Robert.
One idea to make the e2e also run on the Alibaba infrastructure would be to
ensure that e2e tests clean up after they have run. Do we know which e2e
tests don't do this properly?
Cheers,
Till
On Thu, May 14, 2020 at 8:38 AM Robert Metzger wrote:
> Hi all,
>
> tl;d
Thanks for bring this discussion up, Till.
+1 for guaranteeing API + binary compatibility for @PublicEvolving classes
for bug fix releases.
I have a bit concern on the compatibility of @Public classes though.
According to the JavaDoc, @Public requires classes to be stable
across minor releases (x
Thanks Till for starting this discussion.
+1 for enabling the japicmp-maven-plugin for @PublicEvolving for bug fix
releases.
Bug fix should just be user imperceptible bug fix. Should not affect API
and binary compatibility.
And even PublicEvolving api change for "y" release, we should expose it i
So this is a terminology problem.
*Flink does not follow the major.minor.patch naming scheme.*
x.y.z is minor, x.y.0 is major, x.0.0 has no name as of right now.
Public API's may be broken in x.0.0, and Till is not suggesting to
change this.
What he is suggesting is that:
Public APIs may onl
I also like the proposal for keeping the binary compatibility of
@PublicEvolving for bugfix releases.
As for the @Public classes I think the current guarantees are good enough.
Best,
Dawid
On 14/05/2020 10:49, Jingsong Li wrote:
> Thanks Till for starting this discussion.
>
> +1 for enabling th
Roughly speaking, I see the following problematic areas (I have initially
tried running the E2E tests on those machines):
a) e2e tests starting Docker images (including Kubernetes). Since the tests
on the Ali infra are running in docker themselves, we need to adjust the
test scripts (which is not
Thanks for starting this discussion!
I agree turn on japicmp on PublicEvolving among bugfix releases is a nit
win.
@Xintong Song I think @Public guarantee is good
enough, the problem is a reachable 2.0 plan.
My concern is more on classes that have no annotation but our developers
regard as "som
Gary Yao created FLINK-17687:
Summary: Collect TaskManager logs in Mesos Jepsen Tests
Key: FLINK-17687
URL: https://issues.apache.org/jira/browse/FLINK-17687
Project: Flink
Issue Type: Improvemen
Sorry for the confusion. @Public classes are guaranteed to be stable
between releases x.y.z and x.u.v (minor and bug fix release; naming is
indeed a bit off here) and we can break it with major releases (x.0.0 and
y.0.0).
@Tison I would like to make what to include in the public API, hence what
to
roland created FLINK-17688:
--
Summary: Support consuming Kinesis' enhanced fanout for
flink-connector-kinesis
Key: FLINK-17688
URL: https://issues.apache.org/jira/browse/FLINK-17688
Project: Flink
I
Thanks for the clarification.
+1 for keeping the current guarantees for @Public.
Thank you~
Xintong Song
On Thu, May 14, 2020 at 6:07 PM Till Rohrmann wrote:
> Sorry for the confusion. @Public classes are guaranteed to be stable
> between releases x.y.z and x.u.v (minor and bug fix release;
Jark Wu created FLINK-17689:
---
Summary: Add integeration tests for Debezium and Canal formats
Key: FLINK-17689
URL: https://issues.apache.org/jira/browse/FLINK-17689
Project: Flink
Issue Type: Sub-t
I just want to throw in that we also need to rethink our policy towards
using @PublicEvolving.
We often introduce this easily (for every new feature) and rarely (almost
never) upgrade it to @Public. This kind of leads the idea behind stable API
guarantees ad absurdum.
I would suggest that we make
The parameter "state.backend.fs.memory-threshold" decides when a state will
become a file and when it will be stored inline with the metadata (to avoid
excessive amounts of small files).
By default, this threshold is 1K - so every state above that size becomes a
file. For many cases, this threshol
Igal Shilman created FLINK-17690:
Summary: Python function wrapper omits docstr
Key: FLINK-17690
URL: https://issues.apache.org/jira/browse/FLINK-17690
Project: Flink
Issue Type: Improvement
freezhan created FLINK-17691:
Summary: Failed to sinkKafka with exactly once mode when
transactional.id too long
Key: FLINK-17691
URL: https://issues.apache.org/jira/browse/FLINK-17691
Project: Flink
Robert Metzger created FLINK-17692:
--
Summary:
"flink-end-to-end-tests/test-scripts/hadoop/yarn.classpath" present after
building Flink
Key: FLINK-17692
URL: https://issues.apache.org/jira/browse/FLINK-17692
Jark Wu created FLINK-17693:
---
Summary: Add createTypeInformation to DynamicTableSink#Context
Key: FLINK-17693
URL: https://issues.apache.org/jira/browse/FLINK-17693
Project: Flink
Issue Type: Sub-t
Hi Gyula,
thanks for proposing this extension. I can see that such a feature could be
helpful.
However, I wouldn't consider the management of multiple clusters core to
Flink. Managing a single cluster is already complex enough and given the
available community capacity I would rather concentrate
I cannot say much about the concrete value but if our users have problems
with the existing default values, then it makes sense to me to change it.
One thing to check could be whether it is possible to provide a meaningful
exception in case that the state size exceeds the frame size. At the
moment
Stephan Ewen created FLINK-17694:
Summary: Wrong min-length check in SimpleVersionedSerialization
Key: FLINK-17694
URL: https://issues.apache.org/jira/browse/FLINK-17694
Project: Flink
Issue
Stephan Ewen created FLINK-17695:
Summary: Simplify SourceOperator by using a utility
SimpleVersionedListState
Key: FLINK-17695
URL: https://issues.apache.org/jira/browse/FLINK-17695
Project: Flink
Stephan Ewen created FLINK-17696:
Summary: Support eager initialization of operators with
OperatorEventDispatcher
Key: FLINK-17696
URL: https://issues.apache.org/jira/browse/FLINK-17696
Project: Flin
Robert Metzger created FLINK-17697:
--
Summary: JDK11: JdbcFullTest.testEnrichedClassCastException:
expected:<[java.lang.String cannot be cast to java.lang.Double], field index:
3, fi...> but was:<[class java.lang.String
Key: FLIN
Timo Walther created FLINK-17698:
Summary: Support for comparing instance of all types
Key: FLINK-17698
URL: https://issues.apache.org/jira/browse/FLINK-17698
Project: Flink
Issue Type: Bug
Stephan Ewen created FLINK-17699:
Summary: Reduce scope for SourceOperator arguments and initialize
more eagerly
Key: FLINK-17699
URL: https://issues.apache.org/jira/browse/FLINK-17699
Project: Flink
shuiqiangchen created FLINK-17700:
-
Summary: The callback client of JavaGatewayServer should run in a
daemon thread
Key: FLINK-17700
URL: https://issues.apache.org/jira/browse/FLINK-17700
Project: Fli
Stephan Ewen created FLINK-17701:
Summary: Exclude jdk:tools dependency from all Hadoop dependencies
for Java 9+ compatibility
Key: FLINK-17701
URL: https://issues.apache.org/jira/browse/FLINK-17701
P
TL;DR: I have some reservations but tend to be +1 for the proposal,
meanwhile suggest we have a more thorough solution in the long run.
Please correct me if I'm wrong, but it seems the root cause of the issue is
too many small files generated.
I have some concerns for the case of session cluster
Tend to be not in favor of this proposal as union state is somewhat abused in
several popular source connectors (e.g. kafka), and increasing this value could
lead to JM OOM when sending tdd from JM to TMs with large parallelism.
After we collect union state and initialize the map list [1], we al
Stephan Ewen created FLINK-17702:
Summary: OperatorCoordinators must be notified of tasks cancelled
as part of failover
Key: FLINK-17702
URL: https://issues.apache.org/jira/browse/FLINK-17702
Project:
Nico Kruber created FLINK-17703:
---
Summary: Default execution command fails due 'benchmark' profile
being inactive
Key: FLINK-17703
URL: https://issues.apache.org/jira/browse/FLINK-17703
Project: Flink
Nico Kruber created FLINK-17704:
---
Summary: Allow running specific benchmarks from maven directly
Key: FLINK-17704
URL: https://issues.apache.org/jira/browse/FLINK-17704
Project: Flink
Issue Typ
I am interested in applying as a technical writer to the Apache Flink project
in Google Season of Docs. In the project exploration phase I would like to
introduce myself as a potential applicant (when the application opens). I have
experience using several data processing frameworks and have pub
Nico Kruber created FLINK-17705:
---
Summary: Add rat license checks
Key: FLINK-17705
URL: https://issues.apache.org/jira/browse/FLINK-17705
Project: Flink
Issue Type: Sub-task
Component
Nico Kruber created FLINK-17706:
---
Summary: Clarify licensing situation
Key: FLINK-17706
URL: https://issues.apache.org/jira/browse/FLINK-17706
Project: Flink
Issue Type: Sub-task
Comp
Canbin Zheng created FLINK-17707:
Summary: Support configuring replica of Deployment in ZooKeeper
based HA setups
Key: FLINK-17707
URL: https://issues.apache.org/jira/browse/FLINK-17707
Project: Flink
xiemeilong created FLINK-17708:
--
Summary: ScalarFunction has different classloader after submit
task jar to cluster。
Key: FLINK-17708
URL: https://issues.apache.org/jira/browse/FLINK-17708
Project: Flink
Canbin Zheng created FLINK-17709:
Summary: Active Kubernetes integration phase 3 - Advanced Features
Key: FLINK-17709
URL: https://issues.apache.org/jira/browse/FLINK-17709
Project: Flink
Iss
Hequn Cheng created FLINK-17710:
---
Summary: StreamSqlTests.test_execute_sql test is not stable
Key: FLINK-17710
URL: https://issues.apache.org/jira/browse/FLINK-17710
Project: Flink
Issue Type:
Yun Gao created FLINK-17711:
---
Summary: Support hadoop OutputCommitter based file committer
Key: FLINK-17711
URL: https://issues.apache.org/jira/browse/FLINK-17711
Project: Flink
Issue Type: Sub-ta
### Documentation on API compatibility policies
Do we have any formal documentation about the API compatibility policies?
The only things I found are:
- In the release announcement (take 1.10.0 as an example) [1]:
"This version is API-compatible with previous 1.x releases for APIs
annota
+1 for this idea, and I also like Xintong's suggestion to make it explicitly
when the @PublicEvolving API could upgrade to @Public API.
If we have the rule to upgrade API stable level but not define the clear
timeline, I'm afraid not everyone have the enthusiasm to upgrade this.
The minor sugges
+1 for "API + binary compatibility for @PublicEvolving classes for all bug
fix
releases in a minor release (x.y.z is compatible to x.y.u)"
This @PublicEnvolving would then be a hard limit to changes.
So it's important to rethink the policy towards using it, as Stephan
proposed.
I think any Flink
Tzu-Li (Gordon) Tai created FLINK-17712:
---
Summary: Upgrade Flink version to 1.10.1 in Stateful Functions
Key: FLINK-17712
URL: https://issues.apache.org/jira/browse/FLINK-17712
Project: Flink
Robert Metzger created FLINK-17713:
--
Summary: "Stateful stream job upgrade end-to-end test" fails
Key: FLINK-17713
URL: https://issues.apache.org/jira/browse/FLINK-17713
Project: Flink
Issue
Zhu Zhu created FLINK-17714:
---
Summary: Support custom RestartBackoffTimeStrategy
Key: FLINK-17714
URL: https://issues.apache.org/jira/browse/FLINK-17714
Project: Flink
Issue Type: Wish
Co
Danny Chen created FLINK-17715:
--
Summary: Supports create function in SQL-CLI
Key: FLINK-17715
URL: https://issues.apache.org/jira/browse/FLINK-17715
Project: Flink
Issue Type: New Feature
Timo Walther created FLINK-17716:
Summary: Remove deprecated RowData.get and similar methods
Key: FLINK-17716
URL: https://issues.apache.org/jira/browse/FLINK-17716
Project: Flink
Issue Type:
Danny Chen created FLINK-17717:
--
Summary: Throws for DDL create system temporary function with
composite table path
Key: FLINK-17717
URL: https://issues.apache.org/jira/browse/FLINK-17717
Project: Flink
Jingsong Lee created FLINK-17718:
Summary: Integrate avro to file system connector
Key: FLINK-17718
URL: https://issues.apache.org/jira/browse/FLINK-17718
Project: Flink
Issue Type: Sub-task
56 matches
Mail list logo
