Dong Lin created FLINK-22915:
Summary: Extend Flink ML API to support Estimator/Transformer DAG
Key: FLINK-22915
URL: https://issues.apache.org/jira/browse/FLINK-22915
Project: Flink
Issue Type:
Qingsheng Ren created FLINK-22914:
-
Summary: Use Kafka New Source in Table/SQL connector
Key: FLINK-22914
URL: https://issues.apache.org/jira/browse/FLINK-22914
Project: Flink
Issue Type: New
> hybrid sounds to me more like the source would constantly switch back and
forth
Initially, the focus of hybrid source is more like a sequenced chain.
But in the future it would be cool that hybrid sources can intelligently
switch back and forth between historical data source (like Iceberg) and
I can see the benefits of control flow. E.g., it might help the old (and
inactive) FLIP-17 side input. I would suggest that we add more details of
some of the potential use cases.
Here is one mismatch with using control flow for dynamic config. Dynamic
config is typically targeted/loaded by one sp
+1 on separating the effort into two steps:
1. Introduce a common control flow framework, with flexible interfaces
for generating / reacting to control messages for various purposes.
2. Features that leverating the control flow can be worked on
concurrently
Meantime, keeping collectin
Dian Fu created FLINK-22913:
---
Summary: Support Python UDF chaining in Python DataStream API
Key: FLINK-22913
URL: https://issues.apache.org/jira/browse/FLINK-22913
Project: Flink
Issue Type: Improv
Dian Fu created FLINK-22912:
---
Summary: Support state ttl in Python DataStream API
Key: FLINK-22912
URL: https://issues.apache.org/jira/browse/FLINK-22912
Project: Flink
Issue Type: Improvement
Dian Fu created FLINK-22911:
---
Summary: Align FLIP-136 (Improve interoperability between
DataStream and Table API) in PyFlink Table API
Key: FLINK-22911
URL: https://issues.apache.org/jira/browse/FLINK-22911
I think being able to specify fine grained resource requirements without
having to change the codes and recompile the job is indeed a good idea. It
definitely improves the usability.
However, this requires more careful designs, which probably deserves a
separate thread. I'd be good to have that di
Very thanks Jiangang for bringing this up and very thanks for the discussion!
I also agree with the summarization by Xintong and Jing that control flow seems
to be
a common buidling block for many functionalities and dynamic configuration
framework
is a representative application that frequentl
@Wenlong
After another consideration, the config option approach I mentioned
above might not be appropriate. The resource requirements for SSG
should be a job level configuration and should no be set in the
flink-conf.
I think we can define a JSON format, which would be the ResourceSpecs
mapped by
Thanks for the feedbacks, Xintong and Wenlong!
@Wenlong
I think that is a good idea, adjust the resource without re-compiling
the job will facilitate the tuning process.
We can define a pattern "slot-sharing-group.resource.{ssg name}"
(welcome any proposal for the prefix naming) for the resource s
Yingjie Cao created FLINK-22910:
---
Summary: ShuffleMaster enhancement for pluggable shuffle service
framework
Key: FLINK-22910
URL: https://issues.apache.org/jira/browse/FLINK-22910
Project: Flink
I'm big +1 for this feature.
1. Limit the input qps.
2. Change log level for debug.
in my team, the two examples above are needed
JING ZHANG 于2021年6月8日周二 上午11:18写道：
> Thanks Jiangang for bringing this up.
> As mentioned in Jiangang's email, `dynamic configuration framework`
> provides ma
Hello,
My username is Senhong Liu (senhong...@gmail.com) and I want to apply
for permission to propose a FLIP.
Anyone who can help me? THX!
Best,
Senhong
Thanks Jiangang for bringing this up.
As mentioned in Jiangang's email, `dynamic configuration framework`
provides many useful functions in Kuaishou, because it could update job
behavior without relaunching the job. The functions are very popular in
Kuaishou, we also see similar demands in maillist
Jingsong Lee created FLINK-22909:
Summary: Supports change log inputs for event time operators
Key: FLINK-22909
URL: https://issues.apache.org/jira/browse/FLINK-22909
Project: Flink
Issue Typ
Thanks Yangze for the flip, it is great for users to be able to declare the
fine-grained resource requirements for the job.
I have one minor suggestion: can we support setting resource requirements
by configuration? Currently most of the config options in execution config
can be configured by conf
Xintong Song created FLINK-22908:
Summary:
FileExecutionGraphInfoStoreTest.testPutSuspendedJobOnClusterShutdown fails on
azure
Key: FLINK-22908
URL: https://issues.apache.org/jira/browse/FLINK-22908
Thanks Xintong Song for the detailed supplement. Since flink is
long-running, it is similar to many services. So interacting with it or
controlling it is a common desire. This was our initial thought when
implementing the feature. In our inner flink, many configs used in yaml can
be adjusted by dyn
Hi!
Currently Flink File Source relies on a Set pathsAlreadyProcessed in
SplitEnumerator to decide which file has been processed and avoids
reprocessing files if a file is already in this set. However this set could
be ever growing and ultimately exceed memory size if there are new files
continuou
Ryan Darling created FLINK-22907:
Summary: SQL Client queries fails on select statement
Key: FLINK-22907
URL: https://issues.apache.org/jira/browse/FLINK-22907
Project: Flink
Issue Type: Bug
Piotr, David, and Arvid, we've had an expansive discussion but ultimately
the proposal is narrow. It is:
1. When a watermark arrives at the sink operator, tell the sink function.
2. When the sink operator is idled, tell the sink function.
With these enhancements, we will significantly improve cor
Sorry for joining the party so late, but it's such an interesting FLIP with
a huge impact that I wanted to add my 2 cents. [1]
I'm mirroring some basic question from the PR review to this thread because
it's about the name:
We could rename the thing to ConcatenatedSource(s), SourceSequence, or
sim
Seth Wiesman created FLINK-22906:
Summary: Add build time to Flink documentation
Key: FLINK-22906
URL: https://issues.apache.org/jira/browse/FLINK-22906
Project: Flink
Issue Type: Improvement
liuyan created FLINK-22905:
--
Summary: Versioned Table's SQL Script was missing a "," at Line 7
which yields Could not execute SQL statement ERROR
Key: FLINK-22905
URL: https://issues.apache.org/jira/browse/FLINK-22905
Piotr Nowojski created FLINK-22904:
--
Summary: Performance regression on 25.05.2020 in
mapRebalanceMapSink
Key: FLINK-22904
URL: https://issues.apache.org/jira/browse/FLINK-22904
Project: Flink
smith jayden created FLINK-22903:
Summary: Code of method xxx of class "StreamExecCalc$1248" grows
beyond 64 KB
Key: FLINK-22903
URL: https://issues.apache.org/jira/browse/FLINK-22903
Project: Flink
Hi,
Thanks Tianxin and 周瑞' for reporting and tracking down the problem. Indeed
that could be the reason behind it. Have either of you already created a
JIRA ticket for this bug?
> Concerning the required changing of the UID of an operator Piotr, is this
a known issue and documented somewhere? I f
Arvid Heise created FLINK-22902:
---
Summary: Port KafkaSink to FLIP-143
Key: FLINK-22902
URL: https://issues.apache.org/jira/browse/FLINK-22902
Project: Flink
Issue Type: Improvement
Co
Jingsong Lee created FLINK-22901:
Summary: Introduce getChangeLogUpsertKeys in FlinkRelMetadataQuery
Key: FLINK-22901
URL: https://issues.apache.org/jira/browse/FLINK-22901
Project: Flink
Iss
Thanks Xintong for the summary,
I'm big +1 for this feature.
Xintong's summary for Table/SQL's needs is correct.
The "custom (broadcast) event" feature is important to us
and even blocks further awesome features and optimizations in Table/SQL.
I also discussed offline with @Yun Gao several times
One more idea for the bot. Could we have a label to exclude certain tickets
from the life-cycle?
I'm thinking about long-term tickets such as improving DataStream to
eventually replace DataSet. We would collect ideas over the next couple of
weeks without any visible progress on the implementation.
Hi Eron,
you either have very specific use cases in mind or have a misconception
about idleness in Flink with the new sources. The basic idea is that you
have watermark generators only at the sources and the user supplies them.
As a source author, you have no option to limit that. Here a bit of
ba
bigdataf created FLINK-22900:
Summary: flink 1.11.2 fileSystem source table read fileSystem
sink table path multi-partition error
Key: FLINK-22900
URL: https://issues.apache.org/jira/browse/FLINK-22900
P
Thanks for starting this discussion Wenhao. I've given you permission to
create a FLIP.
Cheers,
Till
On Sat, Jun 5, 2021 at 9:48 AM Wenhao Ji wrote:
> Hi everyone,
>
> Currently, the "transactional.id"s of the Kafka producers in
> FlinkKafkaProducer are generated based on the task name. This me
Jingsong Lee created FLINK-22899:
Summary: ValuesUpsertSinkFunction needs to use global upsert
Key: FLINK-22899
URL: https://issues.apache.org/jira/browse/FLINK-22899
Project: Flink
Issue Typ
Jingsong Lee created FLINK-22898:
Summary: HiveParallelismInference limit return wrong parallelism
Key: FLINK-22898
URL: https://issues.apache.org/jira/browse/FLINK-22898
Project: Flink
Issue
zhengjiewen created FLINK-22897:
---
Summary: FlinkSQL1.12 Sink to Hive with diffrent parallelism will
due to produce many small files
Key: FLINK-22897
URL: https://issues.apache.org/jira/browse/FLINK-22897
39 matches
Mail list logo
