Re: Flink V1 to V2 state migration

2024-11-08 Thread Galen Warren
Yes, I'd like to second that it would be very helpful to have a way to migrate state from V1 to V2. On Thu, Nov 7, 2024 at 12:11 PM David Radley wrote: > Hello, > At Flink Forward I learnt that Fink V1 state could not be migrated to V2. > I think this would be a big migration inhibitor for curre

Re: [jira] [Created] (FLINK-34696) GSRecoverableWriterCommitter is generating excessive data blobs

2024-03-15 Thread Galen Warren
Yes, the issue is recoverability. However, there is one thing you can do. Create a separate bucket in GCP for temporary files and initialize the filesystem configuration (via FileSystem.initialize) with *gs.writer.temporary.bucket.name * set to the name of th

Re: [ANNOUNCE] Apache Flink Stateful Functions Release 3.3.0 released

2023-09-19 Thread Galen Warren
Thanks Martijn! On Tue, Sep 19, 2023 at 6:38 AM Martijn Visser wrote: > The Apache Flink community is excited to announce the release of > Stateful Functions 3.3.0! > > Stateful Functions is a cross-platform stack for building Stateful > Serverless applications, making it radically simpler to de

Re: [DISSCUSS] Kubernetes Operator Flink Version Support Policy

2023-09-05 Thread Galen Warren
Sounds good to me, thanks. On Tue, Sep 5, 2023, 8:12 AM Gyula Fóra wrote: > Hi All! > > @Maximilian Michels has raised the question of Flink > version support in the operator before the last release. I would like to > open this discussion publicly so we can finalize this before the next > relea

Re: [DISCUSS] Status of Statefun Project

2023-08-18 Thread Galen Warren
; first place. > > re: paths for locating StateFun artifacts. > Can this be solved by simply passing in the path to the artifacts? As well > as the image tag for the locally build base StateFun image. They could > probably be environment variables. > > Cheers, > Gordon > &g

Re: [DISCUSS] Status of Statefun Project

2023-08-18 Thread Galen Warren
jars + have a local base StateFun image. >2. Run the playground in "local" mode, so that it uses the local base >StateFun image + builds the playground code using copied artifact jars >(instead of pulling from Maven). > > That looks good to me! > > Thanks, &g

Re: [DISCUSS] Status of Statefun Project

2023-08-18 Thread Galen Warren
k the CI manages this workflow. > > Thanks, > Gordon > > On Mon, Aug 14, 2023 at 10:42 AM Galen Warren > wrote: > > > I created a pull request for this: [FLINK-31619] Upgrade Stateful > > Functions to Flink 1.16.1 by galenwarren · Pull Request #331 · > > apac

Re: [DISCUSS] Status of Statefun Project

2023-08-14 Thread Galen Warren
-statefun-playground. I can submit separate PR s those changes if everyone is on board. Also, should I plan to do the same upgrade to handle Flink 1.17.x? It should be easy to do, especially while the 1.16.x upgrade is fresh on my mind. Thanks. On Fri, Aug 11, 2023 at 6:40 PM Galen Warren wrote:

Re: [DISCUSS] Status of Statefun Project

2023-08-11 Thread Galen Warren
ccurs in Docker containers. Gordon, is there a trick to running the sample code in flink-statefun-playground against yet-unreleased code that I'm missing? Thanks. On Sat, Jun 24, 2023 at 12:40 PM Galen Warren wrote: > Great -- thanks! > > I'm going to be out of town for about a

Re: [VOTE] Release 2.0 must-have work items

2023-07-11 Thread Galen Warren
Regarding SinkV1 vs. SinkV2: Is StreamingFileSink a SinkV1-related interface that is proposed to be removed? In a separate thread, it was discussed how it's important not to remove StreamingFileSink as long as this critical issue with SinkV2 is still outstanding -- https://issues.apache.org/jira/pl

Re: [DISCUSS] Status of Statefun Project

2023-06-24 Thread Galen Warren
> Martijn > > On Tue, Jun 20, 2023 at 2:21 PM Galen Warren > wrote: > >> Thanks. >> >> Martijn, to answer your question, I'd need to do a small amount of work >> to get a PR ready, but not much. Happy to do it if we're deciding to >> res

Re: [DISCUSS] Graduate the FileSink to @PublicEvolving

2023-06-23 Thread Galen Warren
ng is good to graduate. WDYT? > Furthermore, I'd like to have more opinions. All opinions together will > help the community build a mature API graduation process. > > Best regards, > Jing > > On Tue, Jun 20, 2023 at 12:48 PM Galen Warren > wrote: > > > Is th

Re: Async I/O: preserve stream order for requests on key level

2023-06-20 Thread Galen Warren
Hi Juho -- I'm doing something similar. In my case, I want to execute async requests concurrently for inputs associated with different keys but issue them sequentially for any given key. The way I do it is to create a keyed stream and use it as an input to an async function. In this arrangement, al

Re: [DISCUSS] Status of Statefun Project

2023-06-20 Thread Galen Warren
that this is sufficient. > > Best, > Gordon > > On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser > wrote: > >> Let me know if you have a PR for a Flink update :) >> >> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user < >> u...@flink.apac

Re: [DISCUSS] Graduate the FileSink to @PublicEvolving

2023-06-20 Thread Galen Warren
Is this issue still unresolved? https://issues.apache.org/jira/plugins/servlet/mobile#issue/FLINK-30238 Based on prior discussion, I believe this could lead to data loss with FileSink. On Tue, Jun 20, 2023, 5:41 AM Jing Ge wrote: > Hi all, > > The FileSink has been marked as @Experimental[1]

Re: [DISCUSS] Status of Statefun Project

2023-06-08 Thread Galen Warren
. you have to make it clear that your fork does not belong to the > >Apache Flink project. (Trademarks and all that) > > > > Neither should be significant hurdles (there should also be plenty of > > online resources regarding 1), and if you do this then you can freely >

Re: [DISCUSS] Status of Statefun Project

2023-06-06 Thread Galen Warren
ged Martijn to take a look at this thread. > To my knowledge the project hasn't decided anything yet. > > On 27/05/2023 04:05, Galen Warren wrote: > > Ok, I get it. No interest. > > > > If this project is being abandoned, I guess I'll work with my own fork. > Is

Re: [DISCUSS] Status of Statefun Project

2023-05-30 Thread Galen Warren
ijn said, let's be clear on what lies ahead (what option is finally > picked) and how to contribute (if possible) moving forward. > > Regards, > > Salva > > On Sat, May 27, 2023 at 4:05 AM Galen Warren via user < > u...@flink.apache.org> wrote: > >> Ok, I ge

Re: [DISCUSS] Status of Statefun Project

2023-05-26 Thread Galen Warren
Ok, I get it. No interest. If this project is being abandoned, I guess I'll work with my own fork. Is there anything I should consider here? Can I share it with other people who use this project? On Tue, May 16, 2023 at 10:50 AM Galen Warren wrote: > Hi Martijn, since you ope

Re: [DISCUSS] Status of Statefun Project

2023-05-16 Thread Galen Warren
Hi Martijn, since you opened this discussion thread, I'm curious what your thoughts are in light of the responses? Thanks. On Wed, Apr 19, 2023 at 1:21 PM Galen Warren wrote: > I use Apache Flink for stream processing, and StateFun as a hand-off point >> for the rest of the appl

Re: [DISCUSS] Status of Statefun Project

2023-04-19 Thread Galen Warren
> > I use Apache Flink for stream processing, and StateFun as a hand-off point > for the rest of the application. > It serves well as a bridge between a Flink Streaming job and > micro-services. This is essentially how I use it as well, and I would also be sad to see it sunsetted. It works well;

Re: [DISCUSS] Status of Statefun Project

2023-04-17 Thread Galen Warren
Are there any next steps here? On Mon, Apr 3, 2023, 12:46 PM Galen Warren wrote: > Thanks for bringing this up. > > I'm currently using Statefun, and I've made a few small code contributions > over time. All of my PRs have been merged into master and most have been > re

Re: [DISCUSS] Status of Statefun Project

2023-04-03 Thread Galen Warren
Thanks for bringing this up. I'm currently using Statefun, and I've made a few small code contributions over time. All of my PRs have been merged into master and most have been released, but a few haven't been part of a release yet. Most recently, I helped upgrade Statefun to be compatible with Fl

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

2023-02-11 Thread Galen Warren
Regarding this part: @Dong > And Yuxia mentioned earlier in this thread that there are bugs such as FLINK-30238 and FLINK-29459 , which makes it hard to use SinkV2 properly in production. I'll

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

2023-02-06 Thread Galen Warren
Recently, a critical bug with the Unified Sink committer was reported: [FLINK-30238] Unified Sink committer does not clean up state on final savepoint - ASF JIRA (apache.org) . Fabian Paul reported: Hi folks, > > I did some initial investigation

Re: Stateful Functions with Flink 1.15 and onwards

2023-01-19 Thread Galen Warren
I thought I'd check in again here, is there any update? If there's anything I can do to help with releasing statefun 3.3, just let me know. Thanks. On Mon, Nov 28, 2022 at 8:12 AM Galen Warren wrote: > Hi Gordon, just checking in. Is there anything I can do to help here? > >

Re: [jira] [Created] (FLINK-30238) Unified Sink committer does not clean up state on final savepoint

2022-11-30 Thread Galen Warren
Thanks Fabian. Do you know if the legacy StreamingFileSink has the same issues? Not asking you to do research but just wondering if you happen to know. On Tue, Nov 29, 2022 at 8:46 AM Fabian Paul wrote: > Hi folks, > > I did some initial investigation, and the problem seems twofold. > > If no

Re: [jira] [Created] (FLINK-30238) Unified Sink committer does not clean up state on final savepoint

2022-11-29 Thread Galen Warren
This seems scary -- am I interpreting it correctly to mean that unified FileSink doesn't work properly with jobs that need to be stopped-with-savepoints and restarted? Should one use the deprecated StreamingFileSink until this is resolved? On Tue, Nov 29, 2022 at 6:02 AM Fabian Paul (Jira) wrote

Re: Stateful Functions with Flink 1.15 and onwards

2022-11-28 Thread Galen Warren
FLINK/Creating+a+Flink+Stateful+Functions+Release > > On Fri, Nov 4, 2022 at 5:54 AM Galen Warren > wrote: > > > Thanks Gordon. > > > > What is the next step here? > > > > On Thu, Nov 3, 2022 at 1:45 PM Tzu-Li (Gordon) Tai > > wrote: > > >

Re: Stateful Functions with Flink 1.15 and onwards

2022-11-04 Thread Galen Warren
5866 >> - https://issues.apache.org/jira/browse/FLINK-25936 >> - https://issues.apache.org/jira/browse/FLINK-25933 >> >> I'll wait for 24 hours before cutting the release branch for 3.3.x, >> unless anyone raises any objections before that. >> >> Than

Re: Stateful Functions with Flink 1.15 and onwards

2022-11-01 Thread Galen Warren
drive this. > > Thanks, > Gordon > > On Mon, Oct 31, 2022 at 12:17 PM Galen Warren > wrote: > > > Yes, that makes sense. > > > > PR is here: [FLINK-29814][statefun] Change supported Flink version to > > 1.15.2 by galenwarren · Pull Request #319 · apache

Re: Stateful Functions with Flink 1.15 and onwards

2022-10-31 Thread Galen Warren
ink there might still be value in supporting 1.15 since not everyone > upgrades Flink very fast. Hopefully, for Statefun the diff between Flink > 1.15 and 1.16 boils down to changing the Flink dependencies. > > Cheers, > Till > > On Mon, Oct 31, 2022 at 2:06 PM Galen Warren

Re: Stateful Functions with Flink 1.15 and onwards

2022-10-31 Thread Galen Warren
7;ve assigned you to this ticket. The next thing after merging > this PR would be creating a new StateFun release. Once we have merged the > PR, let's check who can help with it the fastest. > > Cheers, > Till > > On Mon, Oct 31, 2022 at 1:10 PM Galen Warren > wrote: &

support Kryo.setRegistrationRequired(false)

2022-10-31 Thread Galen Warren
Hi all, I'd be interested to pick up this existing JIRA ticket, to add support to configure whether Kryo requires registrations for all serialized objects. [FLINK-25993] Option to disable Kryo.setRegistrationRequired(false) - ASF JIRA (apache.org)

Re: Stateful Functions with Flink 1.15 and onwards

2022-10-31 Thread Galen Warren
also no way I can think of >> to use existing native Flink operators like async i/o (which when I last >> checked a few years back didn't have access to keyed state) >> >> >> P.S. Please note that there is already a pull request that has something >>

Re: Stateful Functions with Flink 1.15 and onwards

2022-10-26 Thread Galen Warren
Hi Gordon (and others), I'm also using this project for stateful messaging, including messaging among functions. I've contributed a small amount of code in the past and have also enabled Flink 1.15 compatibility in a local fork, so I might be able to help out here. Thanks, Galen On Wed, Oct 26,

Re: [DISCUSS] Bump Kafka to 3.2.1 for 1.16.0

2022-08-09 Thread Galen Warren
I would love to see this issue resolved in 1.16.0. Thanks. On Tue, Aug 9, 2022, 7:50 AM Chesnay Schepler wrote: > Hello, > > The Kafka upgrade in 1.15.0 resulted in a regression > (https://issues.apache.org/jira/browse/FLINK-28060) where offsets are > not committed to Kafka, impeding monitoring

Re: proposal to allow configuration of Statefun exponential backoff

2022-03-22 Thread Galen Warren
No rush, but I just wanted to make sure this didn't fall through the cracks. Thanks! On Fri, Mar 18, 2022 at 4:43 PM Galen Warren wrote: > Currently, when the execution of a stateful function fails, the call is > retried using an exponential backoff scheme -- the initial de

proposal to allow configuration of Statefun exponential backoff

2022-03-18 Thread Galen Warren
Currently, when the execution of a stateful function fails, the call is retried using an exponential backoff scheme -- the initial delay is 10ms (original sync client) or 2ms (async Netty client), and it doubles on every subsequent failure, without limit, until the call succeeds or its budgeted tim

Re: Flink 1.15 Stabilisation Sync skipped tomorrow

2022-03-08 Thread Galen Warren
doing manually testing on my own stuff. On Tue, Mar 8, 2022 at 3:46 AM Johannes Moser wrote: > Thanks so much Gabor and Galen. > > Very much appreciated. > > > On 07.03.2022, at 19:00, Galen Warren wrote: > > > > I just ran through the testing steps on [FLINK-264

Re: Flink 1.15 Stabilisation Sync skipped tomorrow

2022-03-07 Thread Galen Warren
I just ran through the testing steps on [FLINK-26462] Release Testing: Running Python UDF in different Execution Mode - ASF JIRA (apache.org) . Everything worked but I think there might be a typo in the steps to set up the tests, I left a comment

[jira] [Created] (FLINK-26451) Test GCS FileSystem w/ RecoverableWriter manually

2022-03-02 Thread Galen Warren (Jira)
Galen Warren created FLINK-26451: Summary: Test GCS FileSystem w/ RecoverableWriter manually Key: FLINK-26451 URL: https://issues.apache.org/jira/browse/FLINK-26451 Project: Flink Issue Type

Re: possible bug in Statefun Golang SDK related to Context.Caller

2022-02-25 Thread Galen Warren
t;> >> https://github.com/apache/flink-statefun/blob/725202cd69b78442d1287deec000ae2d52da4bda/statefun-sdk-js/src/handler.ts#L240,L244 >> >> On Thu, Feb 24, 2022 at 10:58 PM Galen Warren >> wrote: >> >> > On the Context interface, Caller is defined like this

[jira] [Created] (FLINK-26375) Fix Statefun Golang SDK to return nil from Context.Caller when there is no caller

2022-02-25 Thread Galen Warren (Jira)
Galen Warren created FLINK-26375: Summary: Fix Statefun Golang SDK to return nil from Context.Caller when there is no caller Key: FLINK-26375 URL: https://issues.apache.org/jira/browse/FLINK-26375

Re: possible bug in Statefun Golang SDK related to Context.Caller

2022-02-25 Thread Galen Warren
49,L51 > [2] > > https://github.com/apache/flink-statefun/blob/725202cd69b78442d1287deec000ae2d52da4bda/statefun-sdk-python/statefun/request_reply_v3.py#L115,L122 > [3] > > https://github.com/apache/flink-statefun/blob/725202cd69b78442d1287deec000ae2d52da4bda/statefun-sdk-js/src/handler.ts#L240,L244

possible bug in Statefun Golang SDK related to Context.Caller

2022-02-24 Thread Galen Warren
On the Context interface, Caller is defined like this: // Caller is the caller function instance's Address, if applicable. This is nil // if the message was sent to this function via an ingress. Caller() *Address ... so, in the event a function is invoked from an ingress, as opposed to from anoth

Re: question about StatefunContext in golang Statefun SDK

2022-02-23 Thread Galen Warren
ct (like http.Request) if there is no use case for custom > implementations. > > Thanks for the nice discussion here + contribution :) > > Austin > > > On Wed, Feb 23, 2022 at 11:09 AM Galen Warren > wrote: > >> Thanks, Till. >> >> >>> Is it p

Re: question about StatefunContext in golang Statefun SDK

2022-02-23 Thread Galen Warren
new method would not be breaking. > > Galen, do you want to open a JIRA issue for this proposal? > > Cheers, > Till > > On Wed, Feb 23, 2022 at 12:50 AM Galen Warren > wrote: > > > Yeah, good point. I wasn't considering that someone else might be &

[jira] [Created] (FLINK-26340) Add ability in Golang SDK to create new statefun.Context from existing one, but with a new underlying context.Context

2022-02-23 Thread Galen Warren (Jira)
Galen Warren created FLINK-26340: Summary: Add ability in Golang SDK to create new statefun.Context from existing one, but with a new underlying context.Context Key: FLINK-26340 URL: https://issues.apache.org

Re: question about StatefunContext in golang Statefun SDK

2022-02-22 Thread Galen Warren
ication with adding `WithContext` to the interface > is that it makes it breaking :/ If it were the concrete implementation, > we'd have more flexibility. I'm not sure what guarantees are on those > interfaces though – @sjwies...@gmail.com, wdyt? > > On Tue, Feb 22, 2022 at 4

Re: question about StatefunContext in golang Statefun SDK

2022-02-22 Thread Galen Warren
tefun Context > interface and go with a simpler solution (either the `WithContext` method > or Till's suggestion) to avoid further muddying the possible uses. WDYT? > > Austin > > On Tue, Feb 22, 2022 at 1:14 PM Galen Warren > wrote: > >> One place we could look

Re: question about StatefunContext in golang Statefun SDK

2022-02-22 Thread Galen Warren
verhead to just pass values via the context. Perhaps we should consider > decomposing the statefun context itself so pieces of functionality can be > passed around more easily? > > Best, > Austin > > > [1]: https://pkg.go.dev/net/http#Request.WithContext > > > On Tue, Feb 22,

Re: question about StatefunContext in golang Statefun SDK

2022-02-22 Thread Galen Warren
gt; > > > How do other libraries handle this situation if they offer a custom > > Context type? Maybe @Austin Cawley-Edwards you > > have an opinion on the matter. > > > > Cheers, > > Till > > > > On Mon, Feb 21, 2022 at 7:42 PM Galen Warren > &g

Re: question about StatefunContext in golang Statefun SDK

2022-02-21 Thread Galen Warren
. The statefun.Context would get split into a context.Context and a statefun.Runtime here in order to call the new signature. Thoughts? I'd be happy to take a crack at it. On Mon, Feb 21, 2022 at 12:06 PM Galen Warren wrote: > Was the reason to combine them the desire to have two pa

Re: question about StatefunContext in golang Statefun SDK

2022-02-21 Thread Galen Warren
hy the runtime type went > away). > > Seth > > On Mon, Feb 21, 2022 at 10:32 AM Galen Warren > wrote: > > > Thanks for looking into this! > > > > The issue I think we'd run into with your proposal is that, often, > > libraries use non-export

Re: question about StatefunContext in golang Statefun SDK

2022-02-21 Thread Galen Warren
ntext is not > needed, then there is already the option to unwrap the context.Context and > to extend it with values and then pass on this instance. But maybe this is > no idiomatic Go. Let me know what you think. > > Cheers, > Till > > On Fri, Feb 18, 2022 at 7:01 PM Galen War

Re: question about StatefunContext in golang Statefun SDK

2022-02-18 Thread Galen Warren
8, 2022 at 11:34 AM Galen Warren wrote: > An example of passing it around would be: > > func (f *MyFunc) Invoke(ctx statefun.Context, message statefun.Message) > error { > > logger := NewLogger() > ctx.SetContext(ctxzap.ToContext(ctx, logger)) > > r

Re: question about StatefunContext in golang Statefun SDK

2022-02-18 Thread Galen Warren
:= ctxzap.Extract(ctx) logger.Info(...) return nil } This would also work with further nested calls, so long as the context is passed to them. On Fri, Feb 18, 2022 at 11:23 AM Galen Warren wrote: > Ha, our emails keep passing. > > I've been playing around with options lo

Re: question about StatefunContext in golang Statefun SDK

2022-02-18 Thread Galen Warren
ork. > > > > ``` > > func (s aFunc) Invoke(ctx Context, message Message) error { > > if sCtx, ok := ctx.(*statefun.StatefunContext); ok { > > sCtx.Context = context.WithValue(sCtx.Context, "logger", aLogger) > > } > > // ... > > } > > ``` >

Re: question about StatefunContext in golang Statefun SDK

2022-02-18 Thread Galen Warren
// ... > } > ``` > > Let me know what you all think, > Austin > > > [1]: > > https://github.com/apache/flink-statefun/blob/1dfe226d85fea05a46c8ffa688175b4c0f2d4900/statefun-sdk-go/v3/pkg/statefun/context.go#L66-L73 > > > On Fri, Feb 18, 2022 at 11:03 AM Galen Warren > wrote

Re: question about StatefunContext in golang Statefun SDK

2022-02-18 Thread Galen Warren
tion Galen. If you call the other Go functions, > > then you could also pass the other values as separate arguments to these > > functions, can't you? > > > > Cheers, > > Till > > > > On Fri, Feb 18, 2022 at 3:31 PM Galen Warren > > wrote: > > >

Re: question about StatefunContext in golang Statefun SDK

2022-02-18 Thread Galen Warren
l Rohrmann wrote: > Thanks for the clarification Galen. If you call the other Go functions, > then you could also pass the other values as separate arguments to these > functions, can't you? > > Cheers, > Till > > On Fri, Feb 18, 2022 at 3:31 PM Galen Warren > wrote:

Re: question about StatefunContext in golang Statefun SDK

2022-02-18 Thread Galen Warren
; the Context of function A that is then accessible in a downstream call of > function B? Or would you like to set a value that is accessible once > function A is called again (w/ or w/o the same id)? > > Cheers, > Till > > On Thu, Feb 17, 2022 at 10:59 PM Galen Warren > wrot

Re: question about StatefunContext in golang Statefun SDK

2022-02-17 Thread Galen Warren
FunctionSpec{ FunctionType: statefun.TypeNameFrom( >> "example/my-func"), Function: f }) >> http.Handle("/statefun", builder.AsHandler()) >> _ = http.ListenAndServe(":8000", nil) } >> >> Would this work for you? Or what is the con

Re: question about StatefunContext in golang Statefun SDK

2022-02-17 Thread Galen Warren
dler()) > _ = http.ListenAndServe(":8000", nil) } > > Would this work for you? Or what is the context (pun intended) you are > looking for? > > Seth > > On Wed, Feb 16, 2022 at 4:35 PM Galen Warren > wrote: > > > When stateful functions are invoked, they a

Re: question about StatefunContext in golang Statefun SDK

2022-02-16 Thread Galen Warren
Sorry, that should be context.WithValue, not ctx.WithValue ... On Wed, Feb 16, 2022 at 5:35 PM Galen Warren wrote: > When stateful functions are invoked, they are passed an instance of > statefun.Context, which wraps the context.Context received by the HTTP > request. Is there a

question about StatefunContext in golang Statefun SDK

2022-02-16 Thread Galen Warren
When stateful functions are invoked, they are passed an instance of statefun.Context, which wraps the context.Context received by the HTTP request. Is there any way to customize that context.Context to, say, hold custom values, using ctx.WithValue()? I don't see a way but I wanted to ask. If not,

Re: Statefun async http request via RequestReplyFunctionBuilder

2022-02-09 Thread Galen Warren
e.org/jira/browse/FLINK-25933 > > Cheers, > Till > > On Tue, Feb 8, 2022 at 11:58 PM Galen Warren > wrote: > > > I'm ready to pick this one up, I have some code that's working locally. > > Shall I create a PR? > > > > > > > > On We

Re: Statefun async http request via RequestReplyFunctionBuilder

2022-02-08 Thread Galen Warren
> On Wed, Feb 2, 2022 at 8:55 PM Galen Warren > wrote: > > > Gotcha, thanks. I may be able to work on that one in a couple weeks if > > you're looking for help. > > > > Unrelated question -- another thing that would be useful for me w

Re: Statefun async http request via RequestReplyFunctionBuilder

2022-02-02 Thread Galen Warren
that, if Flink is set to retry on failure, then it will start from the last checkpoint and retry, which is what I wanted to happen. Sorry for the confusion. On Wed, Feb 2, 2022 at 2:55 PM Galen Warren wrote: > Gotcha, thanks. I may be able to work on that one in a couple weeks if > you

Re: Statefun async http request via RequestReplyFunctionBuilder

2022-02-02 Thread Galen Warren
;ve created the following JIRA issue [1] > > > [1] https://issues.apache.org/jira/browse/FLINK-25933 > > On Wed, Feb 2, 2022 at 6:30 PM Galen Warren > wrote: > > > Is it possible to choose the async HTTP transport using > > RequestReplyFunctionBuilder? It looks to me that it is not, but I wanted > to > > double check. Thanks. > > >

Statefun async http request via RequestReplyFunctionBuilder

2022-02-02 Thread Galen Warren
Is it possible to choose the async HTTP transport using RequestReplyFunctionBuilder? It looks to me that it is not, but I wanted to double check. Thanks.

[jira] [Created] (FLINK-25873) Using RocksDB state backend causes java.lang.ClassNotFoundException: org.apache.flink.runtime.state.CheckpointStreamFactory$CheckpointStateOutputStream

2022-01-28 Thread Galen Warren (Jira)
Galen Warren created FLINK-25873: Summary: Using RocksDB state backend causes java.lang.ClassNotFoundException: org.apache.flink.runtime.state.CheckpointStreamFactory$CheckpointStateOutputStream Key: FLINK-25873

[jira] [Created] (FLINK-25790) Support authentication via core-site.xml in GCS FileSystem plugin

2022-01-24 Thread Galen Warren (Jira)
Galen Warren created FLINK-25790: Summary: Support authentication via core-site.xml in GCS FileSystem plugin Key: FLINK-25790 URL: https://issues.apache.org/jira/browse/FLINK-25790 Project: Flink

Re: [jira] [Created] (FLINK-25443) Embedded statefulfunction jakson /java8 date time issue

2021-12-25 Thread Galen Warren
Are you using RequestReplyFunctionBuilder? This may be the same issue I ran into: [FLINK-25197] Using Statefun RequestReplyFunctionBuilder fails with Java 8 date/time type `java.time.Duration` not supported by default: add Module "org.apache.flink.shaded.jackson2.com.fasterxml.jackson.datatype:jac

Re: [jira] [Created] (FLINK-25197) Using Statefun RequestReplyFunctionBuilder fails with Java 8 date/time type `java.time.Duration` not supported by default: add Module "org.apache.flink.shaded.jackso

2021-12-20 Thread Galen Warren
e! I've gone ahead and assigned you > the Jira. And I'll watch the repo to review your pr. > > cheers, > > Seth > > On Mon, Dec 20, 2021 at 8:24 AM Galen Warren > wrote: > > > I just wanted to check in on this, is there any interest in a PR to fix >

Re: [jira] [Created] (FLINK-25197) Using Statefun RequestReplyFunctionBuilder fails with Java 8 date/time type `java.time.Duration` not supported by default: add Module "org.apache.flink.shaded.jackso

2021-12-20 Thread Galen Warren
I just wanted to check in on this, is there any interest in a PR to fix this issue? Looking at the master branch, it still looks to be a problem. Thanks. On Mon, Dec 6, 2021 at 9:52 AM Galen Warren (Jira) wrote: > Galen Warren created FLINK-25

[jira] [Created] (FLINK-25197) Using Statefun RequestReplyFunctionBuilder fails with Java 8 date/time type `java.time.Duration` not supported by default: add Module "org.apache.flink.shaded.jackson2.c

2021-12-06 Thread Galen Warren (Jira)
Galen Warren created FLINK-25197: Summary: Using Statefun RequestReplyFunctionBuilder fails with Java 8 date/time type `java.time.Duration` not supported by default: add Module "org.apache.flink.shaded.jac

Re: Proposal to add Google Cloud Storage FileSystem with RecoverableWriter

2021-02-04 Thread Galen Warren
Hi Till -- this is in response to your message before about my proposal to add GCS FileSystem/RecoverableWriter support. I was only subscribed to the dev digest before, and so I didn't actually get an email I could reply to, so sorry for the one-off email. I'm properly subscribed to the dev list no

Proposal to add Google Cloud Storage FileSystem with RecoverableWriter

2021-01-30 Thread Galen Warren
rking on it have put it on hold, and the last activity was over six months ago. I need this for my own purposes and I have an implementation that I'm working on locally. I'd be interested to contribute this if you'd be interested. Let me know if so and I'll create a Jira ticket. Thanks, Galen Warren

Re: [DISCUSS] Drop Scala 2.11

2020-09-10 Thread Galen Warren
I'd be fine with dropping Scala 2.11. Stateful Functions currently requires 2.11, though, so that would have to be addressed as well. I'm using 2.11 in a production project now but only because of the Stateful Functions dependency. If I could use a newer Scala version that would be fine/preferred.

add support for ScalaPB-based message-payload serialization to Stateful Functions?

2020-09-09 Thread Galen Warren
ed for Java. Would this be something the community would be interested in? Any feedback is welcome. My apologies for the formatting in the ticket, I added hyperlinks to some code that would be involved in the implementation, but the links don't seem to be rendering properly. Thanks, Galen Warren

[jira] [Created] (FLINK-19176) Support ScalaPB as a message payload serializer in Stateful Functions

2020-09-09 Thread Galen Warren (Jira)
Galen Warren created FLINK-19176: Summary: Support ScalaPB as a message payload serializer in Stateful Functions Key: FLINK-19176 URL: https://issues.apache.org/jira/browse/FLINK-19176 Project: Flink

Proposal regarding Queryable State

2018-12-04 Thread Galen Warren
Hi all -- I added an issue for a proposed enhancement to Queryable State, to allow for a transform to be applied to the state information, on the TaskManager, before returning results to the client. More information is in the issue . I'm intereste

[jira] [Created] (FLINK-11055) Allow Queryable State to be transformed on the TaskManager before being returned to the client

2018-12-03 Thread Galen Warren (JIRA)
Galen Warren created FLINK-11055: Summary: Allow Queryable State to be transformed on the TaskManager before being returned to the client Key: FLINK-11055 URL: https://issues.apache.org/jira/browse/FLINK-11055