Hi Chesnay, the following classes also need to be migrated along with the *FlinkContainerTestEnvironment*:
*JobSubmission* *SQLJobSubmission* *ParameterProperty* *FileUtils* We can either put them directly into* flink-connector-test-utils *or into *flink-test-utils *and add a dependency *flink-end-to-end-tests-common -> **flink-test-utils.* What would be your preference? Best, Alex On Mon, May 30, 2022 at 10:59 AM Jing Ge <j...@ververica.com> wrote: > Thanks for starting this discussion. I think it makes sense to make a clear > definition and goal of ITCases and e2e tests. For me, compared to e2e > tests, ITCases are more lightweight and only focus on some specific part of > the e2e pipeline. E2e tests cover the whole pipeline and could be > considered as part of the acceptance test. > > Best regards, > Jing > > On Mon, May 30, 2022 at 10:46 AM Alexander Fedulov < > alexan...@ververica.com> > wrote: > > > Thanks for the clarifications. I agree then that it indeed makes sense to > > move the related classes into connector-test-utils for now and reconsider > > when and if concrete use cases of FlinkContainerTestEnvironment outside > of > > the connectors' scope come up. > > > > > > On Wed, May 25, 2022 at 5:23 PM Chesnay Schepler <ches...@apache.org> > > wrote: > > > > > 1) That's a fair point. So far everything that came out of the > connector > > > e2e framework was unfortunately somehow coupled to connectors > > > conceptually. I wouldn't mind changing that, because as you said this > > > could also be useful for other things, but it needs a more holistic > view > > > over the entire story. > > > That said I'm not sure how usable these things are at this time outside > > > of the connector e2e framework, hence why I'd also consider > > > connector-test-utils as a valid place _for now_. > > > > > > 2) As far as I'm concerned flink-end-to-end-tests-common is a > deprecated > > > module containing legacy code. It primarily contains the java based e2e > > > framework I wrote years back which never got any traction, and in all > > > likelihood will not gain traction in the future. Unfortunately the > > > connector framework was written completely independent from what is > > > already there. > > > > > > 3) Generally I agree. That the environment has a dependency on > > > connector-test-utils was one of the reasons why I proposed to move it > > > there. > > > > > > In general we need to re-think how we organize e2e tests/utils, and > > > whether we even want to differentiate between ITCases and e2e tests, > > > because as is the tests are all over the place (some in the modules, > > > some in flink-e2e-tests). > > > > > > On 25/05/2022 17:06, Alexander Fedulov wrote: > > > > I see. A couple of considerations: > > > > > > > > 1. I would assume FlinkContainerTestEnvironment could be used for > more > > > > things than solely connectors, hence it should ideally not be placed > > into > > > > flink-connector-test-utils > > > > > > > > 2. If we put it and its dependencies into flink-test-utils, what > would > > be > > > > the purpose of the existence of the flink-end-to-end-tests-common > > module, > > > > if we then will also have E2E utilities in flink-test-utils? (other > > than > > > > for segregating stuff that isn't really in a state where we want to > > share > > > > it ) > > > > > > > > 3. If (1.) is true, it seems that ClusterControllable, > TestEnvironment, > > > and > > > > TestEnvironmentSettings are currently misplaced in > > > > flink-connector-test-utils and should be moved together > > > > with FlinkContainerTestEnvironment. > > > > > > > > Let me know what you think. > > > > > > > > Thanks, > > > > Alexander Fedulov > > > > > > > > > > > > On Wed, May 25, 2022 at 3:56 PM Chesnay Schepler <ches...@apache.org > > > > > wrote: > > > > > > > >> I'd be more inclined to move the FlinkContainerTestEnvironment to > > > >> flink-(connector-)test-utils for the time being, because the vast > > > >> majority of stuff in flink-end-to-end-tests-common isn't really in a > > > >> state where we want to share it. > > > >> > > > >> On 25/05/2022 11:55, Alexander Fedulov wrote: > > > >>> Hi everyone, > > > >>> > > > >>> As part of the efforts to externalize the connectors, it became > > evident > > > >>> that some test utilities that reside in Flink will be required in > the > > > >>> external connector repositories. For instance, > > > >> FlinkContainerTestEnvironment > > > >>> and its dependencies reside in the flink-end-to-end-tests-common > > module > > > >> that > > > >>> is currently not published to Maven (there is only an older version > > > from > > > >>> Cloudera available) [1]. > > > >>> > > > >>> I would like to propose to also publish this dependency to Maven. > > What > > > do > > > >>> you think? > > > >>> > > > >>> [1] > https://mvnrepository.com/search?q=flink-end-to-end-tests-common > > > >>> > > > >>> Best, > > > >>> Alexander Fedulov > > > >>> > > > >> > > > > > > > > >
