@Becket One thing that may be non-obvious is that the Configuration class
also defines serialization / persistence logic at the moment. So it needs
to know the set of types it supports. That stands in the way of an
arbitrary generic map type.
@Timo I agree though that it seems a bit inconsistent t
Lord i Will created FLINK-13888:
---
Summary: Translate "How To Contribute" page into Chinese
Key: FLINK-13888
URL: https://issues.apache.org/jira/browse/FLINK-13888
Project: Flink
Issue Type: Tas
richt richt created FLINK-13889:
---
Summary: sql client fetch result oom
Key: FLINK-13889
URL: https://issues.apache.org/jira/browse/FLINK-13889
Project: Flink
Issue Type: Bug
Component
Piotr Nowojski created FLINK-13890:
--
Summary: HiveCatalogUseBlinkITCase failed to get metastore
connection
Key: FLINK-13890
URL: https://issues.apache.org/jira/browse/FLINK-13890
Project: Flink
When computing the values in the JVM process after it started, how would
you deal with values like Max Direct Memory, Metaspace size. native memory
reservation (reduce heap size), etc? All the values that are parameters to
the JVM process and that need to be supplied at process startup?
On Wed, Au
Overall, this is a very nice development that should also simplify the code
base once we deprecate the expression parser!
Regarding method names, I agree with Seth that values/literals should use
something like “lit()”. I also think that for column references we could use
“col()” to make it cle
Chesnay Schepler created FLINK-13891:
Summary: Increment flink-shaded version
Key: FLINK-13891
URL: https://issues.apache.org/jira/browse/FLINK-13891
Project: Flink
Issue Type: Improvemen
Chesnay Schepler created FLINK-13892:
Summary: HistoryServerTest failed on Travis
Key: FLINK-13892
URL: https://issues.apache.org/jira/browse/FLINK-13892
Project: Flink
Issue Type: Improv
Chesnay Schepler created FLINK-13893:
Summary: S3 tests are failing due to missing jaxb dependency
Key: FLINK-13893
URL: https://issues.apache.org/jira/browse/FLINK-13893
Project: Flink
I
Hi Becket,
let me try to clarify some of your questions:
1. For every option, we also needed to think about how to represent it
in a human readable format. We do not want to allow arbitrary nesting
because that would easily allow to bypass the flattened hierarchy of
config options (`session.m
My understanding was that before starting the Flink process we call a
utility which calculates these values. I assume that this utility will do
the calculation based on a set of configured values (process memory, flink
memory, network memory etc.). Assuming that these values don't differ from
the v
I'm fine with `lit()`. Regarding `col()`, I initially suggested `ref()`
but I think Fabian and Dawid liked single char methods for the most
commonly used expressions.
Btw, what is your opinion on the names of commonly used methods such as
`isEqual`, `isGreaterOrEqual`? Are we fine with the cur
lining created FLINK-13894:
--
Summary: Web Ui add log url for subtask of vertex
Key: FLINK-13894
URL: https://issues.apache.org/jira/browse/FLINK-13894
Project: Flink
Issue Type: Improvement
Yu Wang created FLINK-13895:
---
Summary: Client does not exit when bin/yarn-session.sh come fail
Key: FLINK-13895
URL: https://issues.apache.org/jira/browse/FLINK-13895
Project: Flink
Issue Type: Imp
Hi,
IMO, we should define what we would like to optimize for:
1) easy-to-get-started experience or
2) productivity and ease-of-use
While 1) is certainly important, I think we should put more emphasis on
goal 2).
That's why I favor as short as possible names for commonly used methods
like column r
Terry Wang created FLINK-13896:
--
Summary: Scala 2.11 maven compile should target Java 1.8
Key: FLINK-13896
URL: https://issues.apache.org/jira/browse/FLINK-13896
Project: Flink
Issue Type: Bug
I see. Under the assumption of strict determinism that should work.
The original proposal had this point "don't compute inside the TM, compute
outside and supply a full config", because that sounded more intuitive.
On Thu, Aug 29, 2019 at 12:15 PM Till Rohrmann wrote:
> My understanding was tha
I think our goal should be that the configuration is fully specified when
the process is started. By considering the internal calculation step to be
rather validate existing values and calculate missing ones, these two
proposal shouldn't even conflict (given determinism).
Since we don't want to ch
What I forgot to add is that we could tackle specifying the configuration
fully in an incremental way and that the full specification should be the
desired end state.
On Thu, Aug 29, 2019 at 1:33 PM Till Rohrmann wrote:
> I think our goal should be that the configuration is fully specified when
Chesnay Schepler created FLINK-13897:
Summary: OSS FS NOTICE file is placed in wrong directory
Key: FLINK-13897
URL: https://issues.apache.org/jira/browse/FLINK-13897
Project: Flink
Issue
Hi all,
Thank you all for the valuable feedback, let me try to summarize the rough
agreement:
If there is a builder for class A, then A should:
- Have only private ctor -> force usage of the builder
- Be immutable (no setters)
- Have a public static .builder(required params if not too many) m
Till Rohrmann created FLINK-13898:
-
Summary: Migrate restart strategy config constants to
RestartStrategyOptions
Key: FLINK-13898
URL: https://issues.apache.org/jira/browse/FLINK-13898
Project: Flink
limbo created FLINK-13899:
-
Summary: Add SQL DDL for Elasticsearch 5.X version
Key: FLINK-13899
URL: https://issues.apache.org/jira/browse/FLINK-13899
Project: Flink
Issue Type: Improvement
Hi Yu,
Notice that the wiki is still marked as "*Under Discussion*" state.
I think you can update it correspondingly.
Best,
tison.
Yu Li 于2019年8月20日周二 下午10:28写道：
> Sorry for the lag but since we've got a consensus days ago, I started a
> vote thread which will have a result by EOD, thus I'm
Jark Wu created FLINK-13900:
---
Summary: Add a built-in "console" sink to print results in console
Key: FLINK-13900
URL: https://issues.apache.org/jira/browse/FLINK-13900
Project: Flink
Issue Type: N
I converted the mentioned Google doc into a wiki page:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-54%3A+Evolve+ConfigOption+and+Configuration
The core semantics have not changed.
Happy voting,
Timo
On 29.08.19 04:30, Zili Chen wrote:
The design looks good to me.
+1 go ahead!
Bes
Till had already summed it up, but I want to emphasize that Flink as
project only needs to provide #1 (reactive mode) and #3 (active mode, which
necessarily is tied to the cluster manager of choice). The latter would be
needed for Flink jobs to be elastic (in the future), although we may want
to di
In case anyone comes across this thread in the archives,
the FlinkK8sOperator is now available here:
https://github.com/lyft/flinkk8soperator
The community is invited to check it out, provide feedback (use github
issues) or even better, join and contribute to it.
Thomas
On Fri, Nov 2, 2018 at
Hi,
I wanted to propose a new, additional way of configuring execution
parameters that can currently be set only on such objects like
ExecutionConfig, CheckpointConfig and StreamExecutionEnvironment. This
poses problems such as:
* no easy way to configure those from a file
* there is no easy
Hi devs,
GitHub provides a mechanism which is able to require branches to be
up to date before merged[1](point 6). I can see several advantages
enabling it. Thus propose our project to turn on this switch. Below are
my concerns. Looking forward to your insights.
1. Avoid CI failures in pr which f
Hi Timo and Stephan,
Thanks for the detail explanation.
1. I agree that each config should be in a human readable format. My
concern is that the current List looks going a little too far
from what the configuration is supposed to do. They are essentially
creating some Configurable objects instead
Hi!
Huuuge +1 from me, this has been an operational pain for years.
This would also introduce a nice and simple way to extend it in the future
if we need.
Ship it!
Gyula
On Thu, Aug 29, 2019 at 5:05 PM Dawid Wysakowicz
wrote:
> Hi,
>
> I wanted to propose a new, additional way of configuring
What we could also add to make this a bit more generic and extensible is to
create some interfaces for reconfiguring the StreamExecutionEnvironment,
ExecutionConfig etc and let users specify a class the implements the
reconfiguration logic based on the flink configuration.
This could be executed a
Dear Flink developers,
Having difficulty of getting a Flink job started.
The job's uberjar/fat jar is around 400MB, and I need to kick 800+
containers.
The default HDFS replication is 3.
*The Yarn queue is empty, and 800 containers are allocated
almost immediately by Yarn RM.*
It takes v
Thanks everyone for the feedback.
I have updated the document accordingly. Here're the summary of changes:
- clarify the concept of temporary functions, to facilitate deciding
function resolution order
- provide two options to support Hive built-in functions, with the 2nd one
being preferred
- ad
Hi all,
Very thanks Jark for the new scenarios. Based on the these new scenarios, I
think these scenarios and iteration should be able to represent a type of
scenarios that requires broadcasting events.
I also totally agree with Piotr that all the scenarios we have discussed should
be clearly
Hi Zili,
Thanks for the proposal, I had similar confusion in the past with your
point #2.
Force rebase to master before merging can solve some problems, but it also
introduces new problem. Given the CI testing time is quite long (couple of
hours)
now, it's highly possible that before your test whi
Hi Dian,
+1 non-binding
Thanks for driving this!
Best, Wei
> 在 2019年8月29日，09:25，Hequn Cheng 写道：
>
> Hi Dian,
>
> +1
> Thanks a lot for driving this.
>
> Best, Hequn
>
> On Wed, Aug 28, 2019 at 2:01 PM jincheng sun
> wrote:
>
>> Hi Dian,
>>
>> +1, Thanks for your great job!
>>
>> Best,
>
Hi Dian,
+1,
Thanks a lot for driving this.
Best,
Xingbo
> 在 2019年8月30日，上午9:39，Wei Zhong 写道：
>
> Hi Dian,
>
> +1 non-binding
> Thanks for driving this!
>
> Best, Wei
>
>> 在 2019年8月29日，09:25，Hequn Cheng 写道：
>>
>> Hi Dian,
>>
>> +1
>> Thanks a lot for driving this.
>>
>> Best, Hequn
>>
>
Hi Datashov,
We faced similar problems in our production clusters.
Now both lauching and stopping of containers are performed in the main
thread of YarnResourceManager. As containers are launched and stopped one
after another, it usually takes long time to boostrap large jobs. Things
get worse wh
Jark Wu created FLINK-13901:
---
Summary: Documentation links check errors in release-1.9
Key: FLINK-13901
URL: https://issues.apache.org/jira/browse/FLINK-13901
Project: Flink
Issue Type: Bug
+1
Thanks for the great work!
On Fri, 30 Aug 2019 at 10:04, Xingbo Huang wrote:
> Hi Dian,
>
> +1,
> Thanks a lot for driving this.
>
> Best,
> Xingbo
> > 在 2019年8月30日，上午9:39，Wei Zhong 写道：
> >
> > Hi Dian,
> >
> > +1 non-binding
> > Thanks for driving this!
> >
> > Best, Wei
> >
> >> 在 2019年8月
Jingsong Lee created FLINK-13902:
Summary: Can not use index to convert FieldReferenceExpression to
RexNode
Key: FLINK-13902
URL: https://issues.apache.org/jira/browse/FLINK-13902
Project: Flink
Xuefu Zhang created FLINK-13903:
---
Summary: Support Hive version 2.3.6
Key: FLINK-13903
URL: https://issues.apache.org/jira/browse/FLINK-13903
Project: Flink
Issue Type: Improvement
Co
Hi Kurt,
Thanks for your reply!
I find two concerns about the downside from your email. Correct
me if I misunderstanding.
1. Rebase times. Typically commits are independent one another, rebase
just fast-forward changes so that contributors rarely resolve conflicts
by himself. Reviews doesn't get
+1. That would be very helpful.
Best,
Terry Wang
> 在 2019年8月30日，上午10:18，Jark Wu 写道：
>
> +1
>
> Thanks for the great work!
>
> On Fri, 30 Aug 2019 at 10:04, Xingbo Huang wrote:
>
>> Hi Dian,
>>
>> +1,
>> Thanks a lot for driving this.
>>
>> Best,
>> Xingbo
>>> 在 2019年8月30日，上午9:39，Wei Zhon
+1 (non-binding)
Thanks for driving this!
Best Regards,
Yu
On Fri, 30 Aug 2019 at 11:01, Terry Wang wrote:
> +1. That would be very helpful.
> Best,
> Terry Wang
>
>
>
> > 在 2019年8月30日，上午10:18，Jark Wu 写道：
> >
> > +1
> >
> > Thanks for the great work!
> >
> > On Fri, 30 Aug 2019 at 10:04, Xin
Noticing that Flink takes very long inside collect(..) due to Avro
serialization that relies on ReflectDatumWriter & GenericDatumWriter. The
object being serialized here is an Avro object that implements
SpecificRecordBase. It is somewhat about large (~50Kb) and complex.
Looking for a way t
+1， very looking forward this feature in flink 1.10
Yu Li 于2019年8月30日周五 上午11:08写道：
> +1 (non-binding)
>
> Thanks for driving this!
>
> Best Regards,
> Yu
>
>
> On Fri, 30 Aug 2019 at 11:01, Terry Wang wrote:
>
> > +1. That would be very helpful.
> > Best,
> > Terry Wang
> >
> >
> >
> > > 在 201
Biao Liu created FLINK-13904:
Summary: Avoid competition between different rounds of checkpoint
triggering
Key: FLINK-13904
URL: https://issues.apache.org/jira/browse/FLINK-13904
Project: Flink
Hi Zhenghua,
You are right. For per-job cluster, the taskmanagers will be allocated
dynamically by KubernetesResourceManager. For session cluster, we hope
taskmangers could be pre-allocated even though it does not work now.
Please navigate to the doc[1] for more details.
Hi Thomas,
We have
Biao Liu created FLINK-13905:
Summary: Separate checkpoint triggering into stages
Key: FLINK-13905
URL: https://issues.apache.org/jira/browse/FLINK-13905
Project: Flink
Issue Type: Sub-task
52 matches
Mail list logo
