Hi Jing, The proposed plugin would be used by Flink application developers, when they are writing their Flink job. It would trigger during compilation/packaging and would look for known incompatibilities, bad practices, or bugs. For instance one cause of frustration for our customers is connector incompatibilities (specifically Kafka and Kinesis) with certain Flink versions. This plugin would be a quick way to update a list of known incompatibilities, bugs, bad practices, so customers get errors during compilation/packaging and not after they've deployed their Flink job.
From what you're saying, the FLIP route might not be the best way to go. We might publish this plugin in our own GitHub namespace/group first, and then get community acknowledgement/support for it. I believe working with the Flink community on this is key as we'd need their support/opinion to do this the right way and reach more Flink users. Thanks Emre On 21/05/2023, 16:48, "Jing Ge" <j...@ververica.com.inva <mailto:j...@ververica.com.inva>LID> wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Hi Emre, Thanks for your proposal. It looks very interesting! Please pay attention that most connectors have been externalized. Will your proposed plug be used for building Flink Connectors or Flink itself? Furthermore, it would be great if you could elaborate features wrt best practices so that we could understand how the plugin will help us. Afaik, FLIP is recommended for improvement ideas that will change public APIs. I am not sure if a new maven plugin belongs to it. Best regards, Jing On Tue, May 16, 2023 at 11:29 AM Kartoglu, Emre <kar...@amazon.co.uk.inva <mailto:kar...@amazon.co.uk.inva>lid> wrote: > Hello all, > > Myself and 2 colleagues developed a Maven plugin (no support for Gradle or > other build tools yet) that we use internally to detect potential issues in > Flink apps at compilation/packaging stage: > > > * Known connector version incompatibilities – so far covering Kafka > and Kinesis > * Best practices e.g. setting operator IDs > > We’d like to make this open-source. Ideally with the Flink community’s > support/mention of it on the Flink website, so more people use it. > > Going forward, I believe we have at least the following options: > > * Get community support: Create a FLIP to discuss where the plugin > should live, what kind of problems it should detect etc. > * We still open-source it but without the community support (if the > community has objections to officially supporting it for instance). > > Just wanted to gauge the feeling/thoughts towards this tool from the > community before going ahead. > > Thanks, > Emre > >
