Bulk storage of protobuf records in files

to just forbid file storage for protobuf messages! Any opinions? All my best, Ryan Skraba [1]: https://issues.apache.org/jira/browse/FLINK-32008 [2]: https://protobuf.dev/programming-guides/techniques/#streaming

Re: [ACCOUNCE] Apache Flink Elasticsearch Connector 3.0.0 released

Excellent news -- welcome to the new era of easier, more timely and more feature-rich releases for everyone! Great job! Ryan On Thu, Nov 10, 2022 at 3:15 PM Leonard Xu wrote: > Thanks Chesnay and Martijn for the great work! I believe the > flink-connector-shared-utils[1] you built wil

RE: Re: [DISCUSS] Contribution of Multi Cluster Kafka Source

possible across multiple clusters if you assign them to the same alignment group? Thanks! Ryan On 2022/06/28 07:21:15 Mason Chen wrote: > Hi all, > > Thanks for the feedback! I'm adding the users, who responded in the user > mailing list, to this thread. > > @Qingsheng - Yes, I

Re: Flink support on Avro1.11

Hello! I've updated the Avro version to the most recent release (1.11.1) in the PR mentioned my Martijn[1] -- there's an end-to-end TPC-DS test that has failed. I'm not sure it's related, but I'm taking a look. All my best, Ryan [1]: https://github.com/apache/flink/

RE: Re: [DISCUSS] Contribution of Multi Cluster Kafka Source

are working with that is public? Could you attach that so we can start looking at it? Let us know if you need anything from us to help move this forward. Thanks! Ryan On 2022/06/27 03:08:13 Qingsheng Ren wrote: > Hi Mason, > > It sounds like an exciting enhancement to the Kafka source

Questions about Flink Stateful Functions Current Capabilities

know of any companies using StateFuns at scale? Thanks! If you need any clarifications I would be happy to oblige. Ryan van Huuksloot Data Developer | Data Platform Engineering | Streaming Capabilities [image: Shopify] <https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email>

Re: [DataStream API] Watermarks not closing TumblingEventTimeWindow with Reduce function

aking time - Thanks! Closing this ticket Ryan van Huuksloot Data Developer | Data Platform Engineering | Streaming Capabilities [image: Shopify] <https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email> On Mon, Apr 4, 2022 at 8:18 AM Arvid Heise wrote: > This sounds

[DataStream API] Watermarks not closing TumblingEventTimeWindow with Reduce function

27;t clear! Thanks! Ryan van Huuksloot Data Developer | Data Platform Engineering | Streaming Capabilities [image: Shopify] <https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email>

Re: DataStream API: Parquet File Format with Scala Case Classes

e = HybridSource.builder(fileSource) > .addSource(kafkaSource) > .build(); DataStream dataStream = env.fromSource(hybridSource, watermarkStrategy, > name) > Let me know if you have any questions! Thanks, Ryan van Huuksloot Data Developer | Data Platform Engineering | Streaming Capabilit

DataStream API: Parquet File Format with Scala Case Classes

have an example of Parquet being used in a HybridSource along with a Kafka Source? Thanks! Ryan van Huuksloot Data Developer | Data Platform Engineering | Streaming Capabilities [image: Shopify] <https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email>

RE: Anomaly detection Apache Flink

Vigo, I mean that the algorithm is a standalone piece of code. There are no examples that I am aware of for running it using Flink. Ryan From: Salvador Vigo Sent: Saturday, April 4, 2020 12:26 AM To: Marta Paes Moreira Cc: Nienhuis, Ryan ; user Subject: RE: [EXTERNAL] Anomaly detection

RE: Anomaly detection Apache Flink

. https://github.com/aws/random-cut-forest-by-aws Ryan From: Marta Paes Moreira Sent: Friday, April 3, 2020 5:25 AM To: Salvador Vigo Cc: user Subject: RE: [EXTERNAL] Anomaly detection Apache Flink CAUTION: This email originated from outside of the organization. Do not click links or open

unsubscribe

unsubscribe

Re: Window Functions and Empty Panes

just fine, here. Thank you again for your time, Kostas and Konstantin. Ryan On Tue, Apr 18, 2017 at 12:07 PM, Kostas Kloudas < k.klou...@data-artisans.com> wrote: > Hi Ryan, > > “A periodic window like this requires the ability to start a timer without > an element and t

Re: Window Functions and Empty Panes

m all ears, and if I'm able to find a solution I'll report it here. Ryan On Mon, Apr 17, 2017 at 9:53 PM, Konstantin Knauf < konstantin.kn...@tngtech.com> wrote: > Hi Ryan, > > afaik there is no way to override this behaviour. > > Did you have a look at the ProcessF

Window Functions and Empty Panes

or a means of overriding this behavior. Alternatively... is there another mean of creating a stream with a periodic output based on each period's inputs and some stored state? Thank you, Ryan

Re: Maintaining Stream Partitioning after Mapping?

Thank you, Chesnay. My hope is to keep things computationally inexpensive, and if I understand you correctly, that is satisfied even with this rekeying. Ryan On Sat, Apr 15, 2017 at 4:22 AM, Chesnay Schepler wrote: > Hello, > > I think if you have multiple keyBy() transformat

Maintaining Stream Partitioning after Mapping?

correctly this is not feasible with Flink, as map(), filter() etc. all output SingleOutputStreamOperator. But I am hoping that I am missing something. Thank you, Ryan

Re: Using standalone single node without HA in production, crazy?

Thank you Jamie and Ufuk both for such helpful answers! I will continue to explore my options and eagerly await out of the box Mesos support. Ryan On Mon, Jul 4, 2016 at 5:05 AM Ufuk Celebi wrote: > On Fri, Jul 1, 2016 at 3:41 PM, Ryan Crumley wrote: > > Questions: > > 1. I

Re: Using standalone single node without HA in production, crazy?

down the road. It sounds like I can't have my cake and eat it too (yet). =) Ryan On Fri, Jul 1, 2016 at 7:22 PM Jamie Grier wrote: > I started to answer these questions and then realized I was making an > assumption about your environment. Do you have a reliable persistent file >

Using standalone single node without HA in production, crazy?

path? Thanks for your time, Ryan