I've tried to enable additional logging for a few hours today. I think something with junit5 is swallowing the logs. I'm using Bazel and junit5. I setup MiniClusterResourceConfiguration using a custom extension. Are there any known issues with Flink and junit5? I can try switching to junit4.
When I've binary searched this issue, this failure happens if my query in step 3 has a join it. If I remove the join, I can remove step 4 and the code still works. I've renamed a bunch of my tables too and the problem still exists. On Tue, Oct 6, 2020, 00:42 Aljoscha Krettek <aljos...@apache.org> wrote: > Hi Dan, > > there were some bugs and quirks in the MiniCluster that we recently fixed: > > - https://issues.apache.org/jira/browse/FLINK-19123 > - https://issues.apache.org/jira/browse/FLINK-19264 > > But I think they are probably unrelated to your case. Could you enable > logging and see from the logs whether the 2) and 3) jobs execute > correctly on the MiniCluster? > > Best, > Aljoscha > > On 06.10.20 08:08, Dan Hill wrote: > > I'm writing a test for a batch job using > MiniClusterResourceConfiguration. > > > > Here's a simple description of my working test case: > > 1) I use TableEnvironment.executeSql(...) to create a source and sink > table > > using tmp filesystem directory. > > 2) I use executeSql to insert some test data into the source tabel. > > 3) I use executeSql to select from source and insert into sink. > > 4) I use executeSql from the same source to a different sink. > > > > When I do these steps, it works. If I remove step 4, no data gets > written > > to the sink. My actual code is more complex than this (has create view, > > join and more tables). This is a simplified description but highlights > the > > weird error. > > > > Has anyone hit issues like this? I'm assuming I have a small code bug in > > my queries that's causing issues. These queries appear to work in > > production so I'm confused. Are there ways of viewing failed jobs or > > queries with MiniClusterResourceConfiguration? > > > > Thanks! > > - Dan > > > >
