Hi everyone,
I've investigated the infect with higher parallelism jobs.
The result shows:
- The size of TDD after serialization become smaller than before.
While I did not meet any issue with Akka framework when the
parallelism set to 6000.
- There was no significant difference regarding the end
+1 (non-binding)
Best,
Dan Zou
+1
Best,
Kurt
On Tue, Apr 14, 2020 at 3:26 PM zoudan wrote:
> +1 (non-binding)
>
> Best,
> Dan Zou
>
>
>
Hello,
I'm working on an implementation of ORC BulkWriter[1]. As of now, I have
the entire implementation in a separate module called "flink-orc-compress"
under "flink-formats" since I'm not entirely sure whether it should go into
the existing ORC modules i.e flink-orc & flink-orc-nohive.
So my q
Thanks for doing this benchmark @Yangze Guo .
The result looks promising. So I would +1 to refactor ExecutionAttemptID
and IntermediateResultPartitionID.
Regarding why 'The size of TDD after serialization become smaller than
before', I guess it's because the new IntermediateResultPartitionIDs ca
On 10.04.20 17:35, Jark Wu wrote:
1) For correctness, it is necessary to perform the watermark generation as
early as possible in order to be close to the actual data
generation within a source's data partition. This is also the purpose of
per-partition watermark and event-time alignment.
Man
Hi Flink devs
Thanks for the feedbacks! Overall, everyone who replied is positive about
creating such a separate repo to host the flink benchmarks.
Since such an action does not necessarily need a vote (as defined by the
project bylaws), I'll proceed to ask PMC's help to create the repo under A
Hi,
Maybe you should use flink-orc. And use orc-core instead of orc-core with
nohive classifier. We can provide nohive version in the future.
Because orc and hive are so close, orc still relies on some classes of hive
currently.
Apache orc with nohive classifier is for create a variant of core an
Hi Xiaogang,
I very much agree with Jark's and Aljoscha's responses.
On 10/04/2020 17:35, Jark Wu wrote:
> Hi Xiaogang,
>
> I think this proposal doesn't conflict with your use case, you can still
> chain a ProcessFunction after a source which emits raw data.
> But I'm not in favor of chaining P
Yadong Xie created FLINK-17130:
--
Summary: Web UI: Enable listing JM Logs and displaying Logs by
Filename
Key: FLINK-17130
URL: https://issues.apache.org/jira/browse/FLINK-17130
Project: Flink
I
Isn't this technically the adoption of a new codebase?
On 14/04/2020 10:06, Yun Tang wrote:
Hi Flink devs
Thanks for the feedbacks! Overall, everyone who replied is positive about
creating such a separate repo to host the flink benchmarks.
Since such an action does not necessarily need a vote
+1 on creating a new repo, and migrating the flink-benchmarks there.
Regarding Chesnay's concern whether this requires a vote:
While the flink-benchmarks is indeed an existing external codebase, I'm not
entirely sure we do need a vote here.
The bylaws state that:
"Adoption of large existing extern
I'm looking into it.
On 10/04/2020 11:27, tison wrote:
Hi guys,
Right now when I click "JavaDocs" in out docsite[1] it jumps to a
page[2] I think is definitely not
out api documentation. Any thoughts?
Best,
tison.
[1] https://ci.apache.org/projects/flink/flink-docs-master/
[2] https://ci.ap
Chesnay Schepler created FLINK-17131:
Summary: Javadocs broken for master
Key: FLINK-17131
URL: https://issues.apache.org/jira/browse/FLINK-17131
Project: Flink
Issue Type: Bug
MING LIU created FLINK-17132:
Summary: Use the latest version of simpleclient_pushgateway to
solve the "Failed to push metrics to PushGateway" problem
Key: FLINK-17132
URL: https://issues.apache.org/jira/browse/FLINK-
Kai Chen created FLINK-17133:
Summary: Support CsvParser Features for CSV format via SQL DDL
Key: FLINK-17133
URL: https://issues.apache.org/jira/browse/FLINK-17133
Project: Flink
Issue Type: New
Jiayi Liao created FLINK-17134:
--
Summary: Wrong logging information in
Kafka010PartitionDiscoverer#L80
Key: FLINK-17134
URL: https://issues.apache.org/jira/browse/FLINK-17134
Project: Flink
Iss
Dian Fu created FLINK-17135:
---
Summary:
PythonCalcSplitRuleTest.testPandasFunctionMixedWithGeneralPythonFunction failed
on master
Key: FLINK-17135
URL: https://issues.apache.org/jira/browse/FLINK-17135
Proj
On a similar note, I just checked that the Flink currently uses orc 1.4.3
in the dependencies. IMO, it is a little outdated. Can we bump the ORC
version to a slightly newer version - maybe 1.5.x or even 1.6.0?
-
Sivaprasanna
On Tue, Apr 14, 2020 at 1:42 PM Jingsong Li wrote:
> Hi,
>
> Maybe you
Aljoscha Krettek created FLINK-17136:
Summary: Rename toplevel DataSet/DataStream section titles
Key: FLINK-17136
URL: https://issues.apache.org/jira/browse/FLINK-17136
Project: Flink
Iss
Benchao Li created FLINK-17137:
--
Summary: Support mini batch for WindowOperator in blink planner
Key: FLINK-17137
URL: https://issues.apache.org/jira/browse/FLINK-17137
Project: Flink
Issue Type
Piotr Nowojski created FLINK-17138:
--
Summary: LocalExecutorITCase.testParameterizedTypes failed on
travis
Key: FLINK-17138
URL: https://issues.apache.org/jira/browse/FLINK-17138
Project: Flink
jinfeng created FLINK-17139:
---
Summary: The sub-plan reuse optimizer can't reuse sub-plan that
from different sql node tree
Key: FLINK-17139
URL: https://issues.apache.org/jira/browse/FLINK-17139
Project: F
+1
Best,
Kurt
On Mon, Apr 13, 2020 at 9:26 PM Rui Li wrote:
> Hi all,
>
> I'd like to start the vote for FLIP-123[1], which has been discussed in
> thread[2].
>
> The vote will be open for 72h, until April 16 2020 01:00 PM UTC, unless
> there's an objection.
>
> [1]
>
> https://cwiki.apache.or
Till Rohrmann created FLINK-17140:
-
Summary:
AsyncLookupJoinITCase.testAsyncJoinTemporalTableOnMultiFieldsWithUdf failed on
Azure
Key: FLINK-17140
URL: https://issues.apache.org/jira/browse/FLINK-17140
+1
Best,
Jingsong Lee
On Tue, Apr 14, 2020 at 9:29 PM Kurt Young wrote:
> +1
>
> Best,
> Kurt
>
>
> On Mon, Apr 13, 2020 at 9:26 PM Rui Li wrote:
>
> > Hi all,
> >
> > I'd like to start the vote for FLIP-123[1], which has been discussed in
> > thread[2].
> >
> > The vote will be open for 72h,
Hi, yes, we can bump orc-core version to a newer.
Best,
Jingsong Lee
On Tue, Apr 14, 2020 at 8:16 PM Sivaprasanna
wrote:
> On a similar note, I just checked that the Flink currently uses orc 1.4.3
> in the dependencies. IMO, it is a little outdated. Can we bump the ORC
> version to a slightly n
Great feature, I kinda like how the dialect / conformance is handled.
+1. also could you please update the FLIP-123 link and add a
summary/description row in the main page[1] please?
Thanks,
Rong
--
[1]
https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
On Tue, Apr 14
Wenlong Lyu created FLINK-17141:
---
Summary: Name of SQL Operator is too long
Key: FLINK-17141
URL: https://issues.apache.org/jira/browse/FLINK-17141
Project: Flink
Issue Type: Improvement
Sivaprasanna Sethuraman created FLINK-17142:
---
Summary: Bump ORC version
Key: FLINK-17142
URL: https://issues.apache.org/jira/browse/FLINK-17142
Project: Flink
Issue Type: Improvemen
I have created a ticket to update the ORC version.
https://issues.apache.org/jira/browse/FLINK-17142
On Tue, Apr 14, 2020 at 8:18 PM Jingsong Li wrote:
> Hi, yes, we can bump orc-core version to a newer.
>
> Best,
> Jingsong Lee
>
> On Tue, Apr 14, 2020 at 8:16 PM Sivaprasanna
> wrote:
>
> > O
Nico Kruber created FLINK-17143:
---
Summary: Blog feed.xml should only contain excerpts, not full
contents
Key: FLINK-17143
URL: https://issues.apache.org/jira/browse/FLINK-17143
Project: Flink
Dawid Wysakowicz created FLINK-17144:
Summary: Input type generalization does not work correctly for
ARRAY and MAP in sql
Key: FLINK-17144
URL: https://issues.apache.org/jira/browse/FLINK-17144
Pr
Thanks for updating the FLIP, Yangze.
If ExternalResourceInfo is a marker interface, then ExternalResourceDriver
could return a Set. This makes is a bit
nicer for the implementor because he can use the concrete subtype.
If we assume that users will always cast the ExternalResourceInfo instance
in
Jingsong Lee created FLINK-17145:
Summary: AggregateRemoveITCase.testAggregateRemove failed in travis
Key: FLINK-17145
URL: https://issues.apache.org/jira/browse/FLINK-17145
Project: Flink
Is
Hey Rong,
Thanks for the reminder. I have updated the main page to add a row for
FLIP-123.
On Tue, Apr 14, 2020 at 11:02 PM Rong Rong wrote:
> Great feature, I kinda like how the dialect / conformance is handled.
>
> +1. also could you please update the FLIP-123 link and add a
> summary/descrip
Hi all,
The voting time for FLIP-105 has passed. I'm closing the vote now.
There were 5 +1 votes, 3 of which are binding:
- Benchao (non-binding)
- Jark (binding)
- Jingsong Li (binding)
- zoudan (non-binding)
- Kurt (binding)
There were no disapproving votes.
Thus, FLIP-105 has been accepted.
Dian Fu created FLINK-17146:
---
Summary: Support conversion between PyFlink Table and Pandas
DataFrame
Key: FLINK-17146
URL: https://issues.apache.org/jira/browse/FLINK-17146
Project: Flink
Issue Ty
Danny Chen created FLINK-17147:
--
Summary: Remove dynamic table options from table source table
digest
Key: FLINK-17147
URL: https://issues.apache.org/jira/browse/FLINK-17147
Project: Flink
Issu
Dian Fu created FLINK-17148:
---
Summary: Support converting pandas dataframe to flink table
Key: FLINK-17148
URL: https://issues.apache.org/jira/browse/FLINK-17148
Project: Flink
Issue Type: Sub-task
Jark Wu created FLINK-17149:
---
Summary: Introduce Debezium format to support reading debezium
changelogs
Key: FLINK-17149
URL: https://issues.apache.org/jira/browse/FLINK-17149
Project: Flink
Issue
Jark Wu created FLINK-17150:
---
Summary: Introduce Canal format to support reading canal changelogs
Key: FLINK-17150
URL: https://issues.apache.org/jira/browse/FLINK-17150
Project: Flink
Issue Type:
Hi Till,
> ExternalResourceDriver could return a Set.
It sounds good.
> then one could make the interface type-safe by changing it to
> public interface RuntimeContext {
> Set
> getExternalResourceInfo(Class externalResourceType);
> }
I think it may not help.
- I think the assumption of "ther
Dawid Wysakowicz created FLINK-17151:
Summary: Align Calcite's and Flink's SYMBOL types
Key: FLINK-17151
URL: https://issues.apache.org/jira/browse/FLINK-17151
Project: Flink
Issue Type:
44 matches
Mail list logo
