+1 for Timo's proposal. Bash scripts can be difficult to maintain and more complex logic can be hard to express with bash. Thus, implementing logic in Java or Python is a good idea and will help to reuse code. I would also favour Java at the moment, because the rest of Flink is also written in Java.
Cheers, Till On Tue, Mar 27, 2018 at 3:54 PM, zhangminglei <18717838...@163.com> wrote: > At this moment, +1 from my side for maintaining bash scripts. > > Mingleizhang > > > 在 2018年3月27日，下午9:42，Piotr Nowojski <pi...@data-artisans.com> 写道： > > > > +1 > > > > I would personally go with Python, but I see a reason to Kostas’s > arguments in favour of Java. Regardless, based on my experience with > maintaining bash scripts, both of those options (Java/python) are much > better in the long run. > > > > Piotrek > > > >> On 27 Mar 2018, at 15:38, Kostas Kloudas <k.klou...@data-artisans.com> > wrote: > >> > >> Hi Timo, > >> > >> Thanks for opening this. > >> > >> I agree that bash is not the best tool for all the reasons that you > mention plus: > >> 1) it is difficult to write re-usable code > >> 2) there are a lot of ways to express the same thing and difficult to > build “best practices” (as it should be > >> in a community project) resulting in a difficulty also to make it a > real community effort. > >> > >> Now on the with what to replace bash, I agree that Java is the best > option. Flink is mainly written in Java > >> and most of its community is already familiar with it. In addition, > there are a lot of utilities already built > >> that can help reduce the boilerplate code that, for example, check logs. > >> > >> The only alternative I can think of it Python, but given that Flink is > a mainly Java project. > >> I would vouch for Java. > >> > >> Cheers, > >> Kostas > >> > >>> On Mar 27, 2018, at 2:00 PM, Timo Walther <twal...@apache.org> wrote: > >>> > >>> Hi everyone, > >>> > >>> after reviewing a bunch of end-to-end tests, I'm wondering if we > should really continue implementing everything in bash scripts. Wouldn't it > be nicer to implement them in Java code that just calls the interfaces of > Flink (e.g. "./bin/flink run" or REST API)? > >>> > >>> Here are some thoughts why I think we should consider that: > >>> > >>> - Implement the tests in a more object oriented way: We could have > classes such as Cluster, LogAnalyzer, WatchdogProcess, and other utility > classes that can make writing tests easier. This also ensure > maintainability in the future because all tools for proper Java coding > style, comments etc. are already in place. > >>> > >>> - Exception handling and type safety: We could catch excpetion more > easily (e.g. if a REST request fails or a process is not there) > >>> > >>> - Debuggability: We could run end-to-end tests in the IDE for > debugging and set breakpoints etc. > >>> > >>> > >>> What do you think? > >>> > >>> > >>> Regards, > >>> > >>> Timo > >>> > >> > > >
