[jira] [Created] (FLINK-37727) java.lang.UnsatisfiedLinkError error on pemja.core.PythonInterpreter$MainInterpreter.initialize()

Shuyi Chen created FLINK-37727: -- Summary: java.lang.UnsatisfiedLinkError error on pemja.core.PythonInterpreter$MainInterpreter.initialize() Key: FLINK-37727 URL: https://issues.apache.org/jira/browse/FLINK-37727

[jira] [Created] (FLINK-37728) Incorrect import in EmbeddedPythonWindowOperator.java

Shuyi Chen created FLINK-37728: -- Summary: Incorrect import in EmbeddedPythonWindowOperator.java Key: FLINK-37728 URL: https://issues.apache.org/jira/browse/FLINK-37728 Project: Flink Issue Type

[jira] [Created] (FLINK-37366) Allow configurable retry for Kafka topic metadata fetch

Shuyi Chen created FLINK-37366: -- Summary: Allow configurable retry for Kafka topic metadata fetch Key: FLINK-37366 URL: https://issues.apache.org/jira/browse/FLINK-37366 Project: Flink Issue

[jira] [Created] (FLINK-37168) TimerRegistrationAction in unregisteredTimers is not cleaned up properly after the timers are registered

Shuyi Chen created FLINK-37168: -- Summary: TimerRegistrationAction in unregisteredTimers is not cleaned up properly after the timers are registered Key: FLINK-37168 URL: https://issues.apache.org/jira/browse/FLINK

[jira] [Created] (FLINK-36874) KafkaSourceBuilder in pyflink did not implement setDeserializer() method

Shuyi Chen created FLINK-36874: -- Summary: KafkaSourceBuilder in pyflink did not implement setDeserializer() method Key: FLINK-36874 URL: https://issues.apache.org/jira/browse/FLINK-36874 Project: Flink

[jira] [Created] (FLINK-36686) allow customizing envs for flink-webhook container

Shuyi Chen created FLINK-36686: -- Summary: allow customizing envs for flink-webhook container Key: FLINK-36686 URL: https://issues.apache.org/jira/browse/FLINK-36686 Project: Flink Issue Type

[jira] [Created] (FLINK-36685) Enable update/create operation on flinkdeployment resource in mutation webhook

Shuyi Chen created FLINK-36685: -- Summary: Enable update/create operation on flinkdeployment resource in mutation webhook Key: FLINK-36685 URL: https://issues.apache.org/jira/browse/FLINK-36685 Project

access to the design doc of FLINK-12477

Hi Devs, can anyone grant access to the design doc of FLINK-12477? Thanks a lot. https://docs.google.com/document/d/1eDpsUKv2FqwZiS1Pm6gYO5eFHScBHfULKmH1-ZEWB4g Shuyi

Re: [ANNOUNCE] Zhu Zhu becomes a Flink committer

Congratulations! On Sat, Dec 14, 2019 at 7:59 AM Rong Rong wrote: > Congrats Zhu Zhu :-) > > -- > Rong > > On Sat, Dec 14, 2019 at 4:47 AM tison wrote: > > > Congratulations!:) > > > > Best, > > tison. > > > > > > OpenInx 于2019年12月14日周六 下午7:34写道： > > > > > Congrats Zhu Zhu! > > > > > > On Sat,

Re: [VOTE] FLIP-79: Flink Function DDL Support (1.10 Release Feature Only)

+1 (binding) On Sat, Nov 9, 2019 at 11:17 PM Kurt Young wrote: > +1 (binding) > > Best, > Kurt > > > On Sun, Nov 10, 2019 at 12:25 PM Peter Huang > wrote: > > > Hi Yu, > > > > Thanks for your reminder about the timeline of delivering the basic > > function DDL in release 1.10. > > As I replied

Re: [ANNOUNCE] Becket Qin joins the Flink PMC

Congratulations! On Mon, Oct 28, 2019 at 11:18 AM Xingcan Cui wrote: > Congratulations, Becket! > > Best, > Xingcan > > > On Oct 28, 2019, at 1:23 PM, Xuefu Z wrote: > > > > Congratulations, Becket! > > > > On Mon, Oct 28, 2019 at 10:08 AM Zhu Zhu wrote: > > > >> Congratulations Becket! > >> >

Re: Timestamp(timezone) conversion bug in non blink Table/SQL runtime

lem to the same > internal function as you. > > Med venlig hilsen / Best regards > Lasse Nedergaard > > > Den 22. jul. 2019 kl. 20.46 skrev Shuyi Chen : > > Hi all, > > Currently, in the non-blink table/SQL runtime, Flink used > SqlFunctions.internalToTimestamp(long v

Timestamp(timezone) conversion bug in non blink Table/SQL runtime

Hi all, Currently, in the non-blink table/SQL runtime, Flink used SqlFunctions.internalToTimestamp(long v) from Calcite to convert event time (in long) to java.sql.Timestamp. However, as discussed in the recent Calcite mailing list (Jul. 19, 2019), SqlFunctions.internalToTimestamp() assumes the in

[jira] [Created] (FLINK-13372) Timestamp conversion bug in non-blink Table/SQL runtime

Shuyi Chen created FLINK-13372: -- Summary: Timestamp conversion bug in non-blink Table/SQL runtime Key: FLINK-13372 URL: https://issues.apache.org/jira/browse/FLINK-13372 Project: Flink Issue

Re: [ANNOUNCE] Jiangjie (Becket) Qin has been added as a committer to the Flink project

Congrats! On Thu, Jul 18, 2019 at 10:21 AM Thomas Weise wrote: > Congrats! > > > On Thu, Jul 18, 2019 at 9:58 AM Richard Deurwaarder > wrote: > > > Congrats Becket! :) > > > > Richard > > > > On Thu, Jul 18, 2019 at 5:52 PM Xuefu Z wrote: > > > > > Congratulation, Becket! At least you're able

Re: [ANNOUNCE] Rong Rong becomes a Flink committer

Congratulations, Rong! On Thu, Jul 11, 2019 at 8:26 AM Yu Li wrote: > Congratulations Rong! > > Best Regards, > Yu > > > On Thu, 11 Jul 2019 at 22:54, zhijiang wrote: > >> Congratulations Rong! >> >> Best, >> Zhijiang >> >> -- >> F

Re: [DISCUSS] FLIP-38 Support python language in flink TableAPI

before embarking for the first > milestone. > > I'm going to comment on other things in the doc. > > [1] > > https://lists.apache.org/thread.html/f6f8116b4b38b0b2d70ed45b990d6bb1bcb33611fde6fdf32ec0e840@%3Cdev.flink.apache.org%3E > > Thomas > > > On Wed, Apr 3

Re: [DISCUSS] FLIP-38 Support python language in flink TableAPI

Thanks a lot for driving the FLIP, jincheng. The approach looks good. Adding multi-lang support sounds a promising direction to expand the footprint of Flink. Do we have plan for adding Golang support? As many backend engineers nowadays are familiar with Go, but probably not Java as much, adding Go

Re: Expose per partition Kafka lag metric in Flink Kafka connector

ping to bump this topic. Can we get some more input here or on the JIRA on the agreement to resolve this issue? Thanks a lot. Shuyi On Mon, Mar 25, 2019 at 8:43 AM Shuyi Chen wrote: > Thanks a lot, Becket. I am sorry that I was out of the loop for the last > few days due to sickness.

Re: Expose per partition Kafka lag metric in Flink Kafka connector

efinitely > something that should be exposed. I replied to the JIRA with some of my > concerns. Do you mind keeping the discussion in the JIRA ticket so it is > easier for future readers to follow the issue? > > Thanks, > > Jiangjie (Becket) Qin > > On Mon, Mar 18, 2019 at 2:1

Expose per partition Kafka lag metric in Flink Kafka connector

Hi all, We found that Flink's Kafka connector does not expose the per-partition Kafka lag. The metric is available in KafkaConsumer after Kafka 0.10.2. And it's an important metric to diagnose which Kafka partition is lagging in production. I've created a JIRA ( https://issues.apache.org/jira/brow

[jira] [Created] (FLINK-11914) Expose a REST endpoint in JobManager to disconnect specific TaskManager

Shuyi Chen created FLINK-11914: -- Summary: Expose a REST endpoint in JobManager to disconnect specific TaskManager Key: FLINK-11914 URL: https://issues.apache.org/jira/browse/FLINK-11914 Project: Flink

[jira] [Created] (FLINK-11912) Expose per partition Kafka lag metric in Flink Kafka consumer

Shuyi Chen created FLINK-11912: -- Summary: Expose per partition Kafka lag metric in Flink Kafka consumer Key: FLINK-11912 URL: https://issues.apache.org/jira/browse/FLINK-11912 Project: Flink

Re: [DISCUSS] Releasing Flink 1.7.2

HI Gordon, Thanks for bringing up the discussion. The following JIRA/PR is almost there, and it's a major/critical issue that blocks us from upgrading to Flink 1.6 or above in production. W/o this fix, a job might go into an infinite resource acquirement loop without failing itself in YARN. FLI

Re: [DISCUSS] Flink Kerberos Improvement

Hi Rong, thanks a lot for the proposal. Currently, Flink assume the keytab is located in a remote DFS. Pre-installing Keytabs statically in YARN node local filesystem is a common approach, so I think we should support this mode in Flink natively. As an optimazation to reduce the KDC access frequenc

Re: [DISCUSS] Flink SQL DDL Design

t;>>>>>> > >> > https://docs.microsoft.com/en-us/sql/relational-databases/partitions/create-partitioned-tables-and-indexes?view=sql-server-2017 > >>>>>>>>> Best, > >>>>>>>>> Jark > >>>>

Re: [DISCUSS] Flink SQL DDL Design

tractor are > >>>>>> expressing computing logic, the TimestampExtractor has no more > >>>> advantage in > >>>>>> SQL scenarios. > >>>>>> > >>>>>> > >>>>>> 6. Partitioning and

Re: [DISCUSS] Flink SQL DDL Design

yFZ_5NBAHYEB1FVmPpP7RgDGBA/edit# > > > > > > Cheers, > > > Jark > > > > > > On Thu, 29 Nov 2018 at 16:13, Shaoxuan Wang > wrote: > > > > > > > Sure Shuyu, > > > > What I hope is that we can reach an agreement on DDL gramma as

Re: [ANNOUNCE] Apache Flink 1.7.0 released

Thanks a lot for the hard work, Till! Shuyi On Sat, Dec 1, 2018 at 4:07 AM Dominik Wosiński wrote: > Thanks Till for being the release manager! > Thanks Everyone and Great Job. > > Best Regards, > Dom. > > pt., 30 lis 2018 o 13:19 vino yang napisał(a): > > > Thanks Till for your great work, al

Re: [DISCUSS] Flink SQL DDL Design

{ > > > > [ VARCHAR ] > > > > | [ BOOLEAN ] > > > > | [ TINYINT ] > > > > | [ SMALLINT ] > > > > | [ INT ] > > > > | [ BIGINT ] > > > > | [ FLOAT ] > &

Re: [DISCUSS] Flink SQL DDL Design

me attributes. This DDL could be used > for batch usecases, ETL, and materializing SQL queries (no time > operations like windows). > > The unified connector API is high on our priority list for the 1.8 > release. I will try to update the document until mid of next week. > > >

Re: [DISCUSS] Flink SQL DDL Design

> > > > > > http://mail-archives.apache.org/mod_mbox/flink-dev/201810.mbox/%3c6676cb66-6f31-23e1-eff5-2e9c19f88...@apache.org%3E > > > > > > > > > On Fri, Nov 2, 2018 at 10:22 AM Bowen Li wrote: > > > > > > > Thanks Shuyi! > > > &g

Re: [DISCUSS] Flink SQL DDL Design

rowse/FLINK-8003 > > > [4] > > > > > > > > > http://mail-archives.apache.org/mod_mbox/flink-dev/201810.mbox/%3c6676cb66-6f31-23e1-eff5-2e9c19f88...@apache.org%3E > > > > > > > > > On Fri, Nov 2, 2018 at 10:22 AM Bowen Li wrote: >

[jira] [Created] (FLINK-10848) Flink's Yarn ResourceManager can allocate too many excess containers

Shuyi Chen created FLINK-10848: -- Summary: Flink's Yarn ResourceManager can allocate too many excess containers Key: FLINK-10848 URL: https://issues.apache.org/jira/browse/FLINK-10848 Project:

[DISCUSS] Flink SQL DDL Design

design considers DDL for table, view, type, library and function. It will be great to get feedback on the design from the community, and align with latest effort in unified SQL connector API [2] and Flink Hive integration [3]. Any feedback is highly appreciated. Thanks Shuyi Chen [1] https

Re: [DISCUSS] Integrate Flink SQL well with Hive ecosystem

also to FLINK-10556[1]. Please kindly review and share > your feedback. > > > Thanks, > Xuefu > > [1] https://issues.apache.org/jira/browse/FLINK-10556 > > -- > Sender:Xuefu > Sent at:2018 Oct 25 (Thu)

Re: [DISCUSS] Integrate Flink SQL well with Hive ecosystem

Welcome to the community and thanks for the great proposal, Xuefu! I think the proposal can be divided into 2 stages: making Flink to support Hive features, and make Hive to work with Flink. I agreed with Timo that on starting with a smaller scope, so we can make progress faster. As for [6], a prop

Re: Creating a slide set for a Flink intro talk

Thanks a lot, Fabian. That's very useful. I made a quick pass and made a few suggestions. I like the use cases/users section. One general comment I have is that the slides seems emphasizing streaming in the first overview slides, but only mention batch here and there not consistently. In my opinion

[jira] [Created] (FLINK-10516) YarnApplicationMasterRunner fail to initialize FileSystem with correct Flink Configuration during setup

Shuyi Chen created FLINK-10516: -- Summary: YarnApplicationMasterRunner fail to initialize FileSystem with correct Flink Configuration during setup Key: FLINK-10516 URL: https://issues.apache.org/jira/browse/FLINK

Re: [DISCUSS] Improvements to the Unified SQL Connector API

gt; Best, Hequn > > > On Thu, Oct 4, 2018 at 4:09 PM Shuyi Chen wrote: > > > Thanks a lot for the proposal, Timo. I left a few comments. Also, it > seems > > the example in the doc does not have the table type (source, sink and > both) > > property anymore. Are you

Re: [DISCUSS] Improvements to the Unified SQL Connector API

Thanks a lot for the proposal, Timo. I left a few comments. Also, it seems the example in the doc does not have the table type (source, sink and both) property anymore. Are you suggesting drop it? I think the table type properties is still useful as it can restrict a certain connector to be only so

Re: [Proposal] Utilities for reading, transforming and creating Streaming savepoints

+1 on the tooling. Also, you mentioned about state bootstrapping problem. Could you please elaborate on how we can leverage the tooling to solve state bootstrapping? I think this is a common problem to stream processing, and it will be great the community can work on it. Thanks. Shuyi On Wed, Aug

[jira] [Created] (FLINK-10187) Fix LogicalUnnestRule to match Correlate/Uncollect correctly

Shuyi Chen created FLINK-10187: -- Summary: Fix LogicalUnnestRule to match Correlate/Uncollect correctly Key: FLINK-10187 URL: https://issues.apache.org/jira/browse/FLINK-10187 Project: Flink

[jira] [Created] (FLINK-10076) Upgrade Calcite dependency to 1.18

Shuyi Chen created FLINK-10076: -- Summary: Upgrade Calcite dependency to 1.18 Key: FLINK-10076 URL: https://issues.apache.org/jira/browse/FLINK-10076 Project: Flink Issue Type: Task

Re: [ANNOUNCE] New committer: Sihua Zhou

Congratulations! On Fri, Jun 22, 2018 at 11:08 AM Matthias J. Sax wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA512 > > Congrats! > > On 6/22/18 10:33 AM, shimin yang wrote: > > Congrats! > > > > On Sat, Jun 23, 2018 at 1:13 AM Chen Qin > > wrote: > > > >> Congrats! > >> > >>> On Jun 2

Re: [ANNOUNCE] New committer Piotr Nowojski

Congratulations! On Fri, Jun 22, 2018 at 12:30 PM Matthias J. Sax wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA512 > > Congrats! > > On 6/22/18 12:28 PM, Stefan Richter wrote: > > Congrats Piotr! > > > >> Am 22.06.2018 um 21:26 schrieb Till Rohrmann > >> : > >> > >> Hi everybody, > >>

Re: [DISCUSS] Adding new interfaces in [Stream]ExecutionEnvironment

> of > > course, would entail that one has to take care of clean up of the > > downloaded resources. But it should be possible to first download the > > resources and create a custom URLClassLoader at startup and then use this > > class loader when calling into the UDF. >

[jira] [Created] (FLINK-9523) Add Kafka examples for Flink Table/SQL API

Shuyi Chen created FLINK-9523: - Summary: Add Kafka examples for Flink Table/SQL API Key: FLINK-9523 URL: https://issues.apache.org/jira/browse/FLINK-9523 Project: Flink Issue Type: Task

Re: Ask for SQL using kafka in Flink

Given the popularity of Flink SQL and Kafka as streaming source, I think we can add some examples of using Kafka[XXX]TableSource in flink-examples/flink-examples-table module. What do you guys think? Cheers Shuyi On Mon, Jun 4, 2018 at 12:57 AM, Timo Walther wrote: > Hi, > > as you can see in c

Re: [SQL] [CALCITE] not applicable function for TIME

I think you might find some context in these 2 PRs in Flink & Calcite respectively: https://issues.apache.org/jira/browse/CALCITE-1987 https://issues.apache.org/jira/browse/FLINK-7934 We have different EXTRACT implementation paths in Calcite and Flink. Hope it helps. On Thu, May 31, 2018 at 7:13

[jira] [Created] (FLINK-9477) Support SQL 2016 JSON functions in Flink SQL

Shuyi Chen created FLINK-9477: - Summary: Support SQL 2016 JSON functions in Flink SQL Key: FLINK-9477 URL: https://issues.apache.org/jira/browse/FLINK-9477 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-9430) Support Casting of Object to Primitive types for Flink SQL UDF

Shuyi Chen created FLINK-9430: - Summary: Support Casting of Object to Primitive types for Flink SQL UDF Key: FLINK-9430 URL: https://issues.apache.org/jira/browse/FLINK-9430 Project: Flink

Re: [DISCUSS] GitBox

+1 :) A lot of projects are already using it. On Wed, May 16, 2018 at 3:40 AM, Chesnay Schepler wrote: > Hello, > > during the discussion about how to better manage pull requests [1] the > topic of GitBox integration came up again. > > This seems like a good

Re: [DISCUSS] Adding new interfaces in [Stream]ExecutionEnvironment

gt; >>>>> > >>>>> +1. This could be very useful for "dynamic" UDF. > >>>>>> > >>>>>> Just to clarify, if I understand correctly, we are tying to use an > >>>>>> ENUM > >>>>>> in

Re: [DISCUSS] Configuration for local recovery

+1 to the proposal. IMO, the current option "ENABLE_FILE_BASED" contains too much implementation details and might confuse the simple users. Having a simple on/off toggle for majority of the users and an advanced option for the experts to do the tuning will definitely make better user experience.

[DISCUSS] Adding new interfaces in [Stream]ExecutionEnvironment

Hi Flink devs, In an effort to support loading external libraries and creating UDFs from external libraries using DDL in Flink SQL, we want to use Flink’s Blob Server to distribute the external libraries in runtime and load those libraries into the user code classloader automatically. However, th

[jira] [Created] (FLINK-9327) Support explicit ROW value constructor in Flink SQL

Shuyi Chen created FLINK-9327: - Summary: Support explicit ROW value constructor in Flink SQL Key: FLINK-9327 URL: https://issues.apache.org/jira/browse/FLINK-9327 Project: Flink Issue Type: Sub

Re: [ANNOUNCE] Two new committers: Xingcan Cui and Nico Kruber

Congratulations! On Tue, May 8, 2018 at 12:18 PM, Dawid Wysakowicz < wysakowicz.da...@gmail.com> wrote: > Congratulations Nico and Xingcan! Well deserved! > > > On 08.05.2018 20:52, Fabian Hueske wrote: > > Hi everyone, > > > > I'm happy to announce that two members of the Flink community accepte

Re: [DISCUSS] Releasing Flink 1.5.0

Hi Aljoscha and Till, I've added 2 PRs to fix and harden the YARN kerberos security for flip-6. I think they should go in for 1.5.0 (particularly FLINK-8286 ). 1) https://github.com/apache/flink/pull/5896 (FLINK-8286

[jira] [Created] (FLINK-9235) Add Integration test for Flink-Yarn-Kerberos integration for flip-6

Shuyi Chen created FLINK-9235: - Summary: Add Integration test for Flink-Yarn-Kerberos integration for flip-6 Key: FLINK-9235 URL: https://issues.apache.org/jira/browse/FLINK-9235 Project: Flink

Re: Support for out-of-the-box external catalog for SQL Client

, > > An HCatalog integration with the Table API/SQL would be great and be > helpful for many users! > > A big +1 to that. > > Thank you, > Fabian > > Shuyi Chen schrieb am Mi., 11. Apr. 2018, 14:36: > > > Thanks a lot, Rong and Peter. > > > > AFAIK, t

[jira] [Created] (FLINK-9171) Flink HCatolog integration

Shuyi Chen created FLINK-9171: - Summary: Flink HCatolog integration Key: FLINK-9171 URL: https://issues.apache.org/jira/browse/FLINK-9171 Project: Flink Issue Type: Task Reporter

[jira] [Created] (FLINK-9170) HCatolog integration with Table/SQL API

Shuyi Chen created FLINK-9170: - Summary: HCatolog integration with Table/SQL API Key: FLINK-9170 URL: https://issues.apache.org/jira/browse/FLINK-9170 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-9161) Support AS STRUCT syntax to create named STRUCT in SQL

Shuyi Chen created FLINK-9161: - Summary: Support AS STRUCT syntax to create named STRUCT in SQL Key: FLINK-9161 URL: https://issues.apache.org/jira/browse/FLINK-9161 Project: Flink Issue Type

Re: Support for out-of-the-box external catalog for SQL Client

Thanks a lot, Rong and Peter. AFAIK, there is a flink hcatalog connector introduced in FLINK-1466 that is added by Fabian. And there is another JIRA in FLINK-1913 to document the use of connector.

Re: [DISCUSS] Implement end-to-end tests in Java

+1 on Timo's proposal. Bash script can be difficult to extend, debug and maintain as the end-to-end tests become larger and more complex. I would favor Java comparing to Bash script. Another option would be to use scala, it's JVM based and work with Java, but at the same time much more concise. O

Re: [DISCUSS] Flink security improvements

t; > it soon. My gut feeling is that the community will pick up the security > > improvements thread once most of the blocking issues are resolved. Please > > bear with us until then. > > > > Cheers, > > Till > > > > On Fri, Mar 9, 2018 at 1:05 AM, Shuyi

[jira] [Created] (FLINK-9059) Add support for unified table source and sink declaration in environment file

Shuyi Chen created FLINK-9059: - Summary: Add support for unified table source and sink declaration in environment file Key: FLINK-9059 URL: https://issues.apache.org/jira/browse/FLINK-9059 Project: Flink

[jira] [Created] (FLINK-9049) Create unified interfaces to configure and instatiate TableSink

Shuyi Chen created FLINK-9049: - Summary: Create unified interfaces to configure and instatiate TableSink Key: FLINK-9049 URL: https://issues.apache.org/jira/browse/FLINK-9049 Project: Flink

[jira] [Created] (FLINK-9015) Upgrade Calcite dependency to 1.17

Shuyi Chen created FLINK-9015: - Summary: Upgrade Calcite dependency to 1.17 Key: FLINK-9015 URL: https://issues.apache.org/jira/browse/FLINK-9015 Project: Flink Issue Type: Task

Re: Flip 6 mesos support

an find more about this effort here: > https://issues.apache.org/jira/browse/FLINK-8970. > > Cheers, > Till > > On Thu, Mar 15, 2018 at 8:45 PM, Shuyi Chen wrote: > > > Hi Till, > > > > This is Shuyi :) Thanks a lot. In FLINK-8562, I already sent a PR to > > resolve the

Re: Flip 6 mesos support

Hi Till, This is Shuyi :) Thanks a lot. In FLINK-8562, I already sent a PR to resolve the issue, your help to take a look will be great. Please let me know what I can help to test the Kerberos authentication, I am decently familiar with the Kerberos and YARN security part in Flink. As a starting

Re: Flip 6 mesos support

Hi Till, have we tested the YARN kerberos integration in flip6? AFAI remember, YARNSessionFIFOSecuredITCase is not functioning (FLINK-8562 ), do we have similar integration test for flip6? Also, Flink yarn kerberos integration in the old deployment

Re: [DISCUSS] Flink security improvements

Ping :) On Wed, Feb 21, 2018 at 7:16 PM, Shuyi Chen wrote: > Hi Eron, thanks a lot for taking a look at the proposal, the comments are > very useful. I've updated the document to address your concerns. Could you > please help take another look, and suggest what the next s

Re: [DISCUSS] Convert main Table API classes into traits

Hi Timo, I am throwing some second thoughts here, as I don't quite see what trait provides over abstract class here for TableEnvironment case. Trait in scala can also have implementation and you can have 'private[flink]' or 'protected' type and method in trait as well. AFAIK, the differences bet

Re: Flink Table 1.5-SNAPSHOT

Maybe you can explain a bit more on what you need. But here is the link to the jar for your convenience, https://drive.google.com/file/d/0BwG71AtfYG0aZk04MHNSMDZiZTlkbUh0dXktMXphZ1c3YTdR/view?usp=sharing. Just rename it and remove the .tmp suffix. On Sat, Feb 24, 2018 at 9:40 AM, Pavel Ciorba wro

Re: [DISCUSS] Flink security improvements

Hi Eron, thanks a lot for taking a look at the proposal, the comments are very useful. I've updated the document to address your concerns. Could you please help take another look, and suggest what the next step is? Highly appreciated. Shuyi On Thu, Feb 15, 2018 at 4:19 AM, Shuyi Chen

[DISCUSS] Flink security improvements

Hi community, I would like to propose a few improvements in Flink security regarding scalability and extensibility. Here is the proposal: https://docs.google.com/document/d/10V7LiNlUJKeKZ58mkR7oVv1t6BrC6TZi3FGf2Dm6-i8/edit?usp=sharing Comments are highly appreciated. Please let me know what the

[jira] [Created] (FLINK-8562) Fix YARNSessionFIFOSecuredITCase

Shuyi Chen created FLINK-8562: - Summary: Fix YARNSessionFIFOSecuredITCase Key: FLINK-8562 URL: https://issues.apache.org/jira/browse/FLINK-8562 Project: Flink Issue Type: Bug

Re: [DISCUSS] Releasing Flink 1.5.0

Hi Aljoscha, can we get this feature in for 1.5.0? We have a lot of internal users waiting for this feature. [FLINK-7923 ] Support accessing subfields of a Composite element in an Object Array type column Thanks a lot Shuyi On Mon, Feb 5, 2018 a

[jira] [Created] (FLINK-8509) Remove SqlGroupedWindowFunction from Flink repo

Shuyi Chen created FLINK-8509: - Summary: Remove SqlGroupedWindowFunction from Flink repo Key: FLINK-8509 URL: https://issues.apache.org/jira/browse/FLINK-8509 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-8508) Remove RexSimplify from Flink repo

Shuyi Chen created FLINK-8508: - Summary: Remove RexSimplify from Flink repo Key: FLINK-8508 URL: https://issues.apache.org/jira/browse/FLINK-8508 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-8507) Upgrade Calcite dependency to 1.16

Shuyi Chen created FLINK-8507: - Summary: Upgrade Calcite dependency to 1.16 Key: FLINK-8507 URL: https://issues.apache.org/jira/browse/FLINK-8507 Project: Flink Issue Type: Task

Re: Flink-Yarn-Kerberos integration

some feedback on potential solutions to those challenges, culminating in a > concrete improvement proposal. > > Thanks > > On Wed, Jan 17, 2018 at 7:29 PM, Shuyi Chen wrote: > > > Ping, any comments? Thanks a lot. > > > > Shuyi > > > > On Wed, Jan 3,

Re: Flink-Yarn-Kerberos integration

Ping, any comments? Thanks a lot. Shuyi On Wed, Jan 3, 2018 at 3:43 PM, Shuyi Chen wrote: > Thanks a lot for the clarification, Eron. That's very helpful. Currently, > we are more concerned about 1) data access, but will get to 2) and 3) > eventually. > > I was thinking

Re: [DISCUSS] Releasing Flink 1.4.1

Can we add this to 1.4.1 release? The flink-yarn-kerberos deployment is not working in 1.4.0. https://issues.apache.org/jira/browse/FLINK-8275 Thanks a lot. On Mon, Jan 15, 2018 at 7:53 AM, vishal wrote: > Hello folks, > > https://issues.apache.org/jira/browse/FLINK-8226 . We would want a

[jira] [Created] (FLINK-8401) Allow subclass to override write-failure behavior in CassandraOutputFormat

Shuyi Chen created FLINK-8401: - Summary: Allow subclass to override write-failure behavior in CassandraOutputFormat Key: FLINK-8401 URL: https://issues.apache.org/jira/browse/FLINK-8401 Project: Flink

[jira] [Created] (FLINK-8397) Support ROW type in CassandraOutputFormat

Shuyi Chen created FLINK-8397: - Summary: Support ROW type in CassandraOutputFormat Key: FLINK-8397 URL: https://issues.apache.org/jira/browse/FLINK-8397 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-8390) Refactor Hadoop kerberos integration test code

Shuyi Chen created FLINK-8390: - Summary: Refactor Hadoop kerberos integration test code Key: FLINK-8390 URL: https://issues.apache.org/jira/browse/FLINK-8390 Project: Flink Issue Type: Bug

Re: Flink-Yarn-Kerberos integration

tell if I misunderstood. > > Eron > > > On Wed, Dec 20, 2017 at 11:21 AM, Shuyi Chen wrote: > > > Hi community, > > > > We are working on secure Flink on YARN. The current Flink-Yarn-Kerberos > > integration will require each container of a job to log in Ke

Flink-Yarn-Kerberos integration

Hi community, We are working on secure Flink on YARN. The current Flink-Yarn-Kerberos integration will require each container of a job to log in Kerberos via keytab every say, 24 hours, and does not use any Hadoop delegation token mechanism except when localizing the container. As I fixed the curr

[jira] [Created] (FLINK-8286) Investigate Flink-Yarn-Kerberos integration for flip-6

Shuyi Chen created FLINK-8286: - Summary: Investigate Flink-Yarn-Kerberos integration for flip-6 Key: FLINK-8286 URL: https://issues.apache.org/jira/browse/FLINK-8286 Project: Flink Issue Type

[jira] [Created] (FLINK-8275) Flink YARN deployment with Kerberos enabled not working

Shuyi Chen created FLINK-8275: - Summary: Flink YARN deployment with Kerberos enabled not working Key: FLINK-8275 URL: https://issues.apache.org/jira/browse/FLINK-8275 Project: Flink Issue Type

[jira] [Created] (FLINK-8039) Support "CREATE TYPE" DDL in Flink SQL

Shuyi Chen created FLINK-8039: - Summary: Support "CREATE TYPE" DDL in Flink SQL Key: FLINK-8039 URL: https://issues.apache.org/jira/browse/FLINK-8039 Project: Flink Issue Type: N

[jira] [Created] (FLINK-8003) Support Calcite's ROW value constructor in Flink SQL

Shuyi Chen created FLINK-8003: - Summary: Support Calcite's ROW value constructor in Flink SQL Key: FLINK-8003 URL: https://issues.apache.org/jira/browse/FLINK-8003 Project: Flink Issue Type

[jira] [Created] (FLINK-7860) Support YARN proxy user in Flink (impersonation)

Shuyi Chen created FLINK-7860: - Summary: Support YARN proxy user in Flink (impersonation) Key: FLINK-7860 URL: https://issues.apache.org/jira/browse/FLINK-7860 Project: Flink Issue Type: New

[jira] [Created] (FLINK-7690) Do not call actorSystem.awaitTermination from the main akka message handling thread

Shuyi Chen created FLINK-7690: - Summary: Do not call actorSystem.awaitTermination from the main akka message handling thread Key: FLINK-7690 URL: https://issues.apache.org/jira/browse/FLINK-7690 Project

[jira] [Created] (FLINK-7658) Support COLLECT Aggregate function in Flink TABLE API

Shuyi Chen created FLINK-7658: - Summary: Support COLLECT Aggregate function in Flink TABLE API Key: FLINK-7658 URL: https://issues.apache.org/jira/browse/FLINK-7658 Project: Flink Issue Type

[jira] [Created] (FLINK-7546) Support SUBMULTISET_OF Operator for Multiset SQL type

Shuyi Chen created FLINK-7546: - Summary: Support SUBMULTISET_OF Operator for Multiset SQL type Key: FLINK-7546 URL: https://issues.apache.org/jira/browse/FLINK-7546 Project: Flink Issue Type

[jira] [Created] (FLINK-7545) Support MEMBER OF Operator for Multiset SQL type

Shuyi Chen created FLINK-7545: - Summary: Support MEMBER OF Operator for Multiset SQL type Key: FLINK-7545 URL: https://issues.apache.org/jira/browse/FLINK-7545 Project: Flink Issue Type: New

  1   2   >