Re: Defining GlobalJobParameters in Flink Unit Testing Harnesses

2021-03-04 Thread Chesnay Schepler
hmm...at a glance I don't anything else that would prevent things from working. I couldn't find a place where we copy the ExecutionConfig, so you should be modifying the very config that is passed to your function. It seems we inevitably have to dive deep into the source / debugging which obje

Re: Defining GlobalJobParameters in Flink Unit Testing Harnesses

2021-03-04 Thread Rion Williams
Thanks Chesnay! I tried giving that a shot but I still wasn't able to access the globalJobParameters from within the open function in my KeyedProcessFunction. You can see the implementation below which I believe should be correct: object CustomProcessFunctionTestHarness { fun forKeyedProcess

Re: Defining GlobalJobParameters in Flink Unit Testing Harnesses

2021-03-04 Thread Chesnay Schepler
The reason why your attempts have failed is that ProcessFunctionTestHarnesses.forKeyedProcessFunction automatically calls open(), thus any mutations on the harness happen too late. I'd suggest to take a look at the implementation of that method and essentially copy the code. You can then call

Re: Defining GlobalJobParameters in Flink Unit Testing Harnesses

2021-03-04 Thread Rion Williams
Absolutely, I think it's gone through quite a few iterations, but this is the current state of it (defined in a @Before function as part of scaffolding out the tests): private lateinit var magicWindowHarness: KeyedOneInputStreamOperatorTestHarness @Before fun init() { magicWindowHarness = Pr

Re: Defining GlobalJobParameters in Flink Unit Testing Harnesses

2021-03-04 Thread Chesnay Schepler
Could you show us how you create test harness? On 3/4/2021 5:13 AM, Rion Williams wrote: Hi all, Early today I had asked a few questions regarding the use of the many testing constructs available within Flink and believe that I have things in a good direction at present. I did run into a spec

Defining GlobalJobParameters in Flink Unit Testing Harnesses

2021-03-03 Thread Rion Williams
Hi all, Early today I had asked a few questions regarding the use of the many testing constructs available within Flink and believe that I have things in a good direction at present. I did run into a specific case that either may not be supported, or just isn't documented well enough for me to det