Hi Dian, thank you for the explanation. I guess this is, at least as far as we know, as good as it gets for an IntelliJ setup for all of Flink. I'll put up a pull request to update the IDE setup guide, but also point out that for specific work on PyFlink, a separate setup with PyCharm is recommended.
Best Ingo On Wed, Aug 18, 2021 at 4:20 PM Dian Fu <dian0511...@gmail.com> wrote: > Hi Ingo, > > There are both Java/Python source codes in PyFlink and I use two IDEs at > the same time: IntelliJ IDEA & PyCharm. > > Regarding ONLY using IntelliJ IDEA for both Python & Java development, I > have tried it and it works just as you said. I have done the following: > - Install Python Plugin > - Mark the module flink-python as a Python module: right click > "flink-python" -> "Open Module Settings", change the Module SDK to Python > - Click file setup.py under flink-python, then it will promote to install a > few Python libraries, just install them > - Run the Python tests under sub-directories of flink-python/pyflink > > Regarding “a) Step (3) will be undone on every Maven import since Maven > sets the SDK for the module”, this has not occurred in my environment for > now. Are there any differences for you regarding the above steps? > > PS: For me, I will still use both IDEs for PyFlink development as it > contains both Java & Python source code under flink-python directory and it > doesn't support configuring both Python & Java SDK for it. However, if > one works > with flink-python occasionally, using only IntelliJ IDEA is a good approach > and should be enough. > > Regards, > Dian > > On Wed, Aug 18, 2021 at 7:33 PM Ingo Bürk <i...@ververica.com> wrote: > > > Hi Dian, > > > > thanks for responding! No, I haven't tried, but I'm aware of it. I don't > > work exclusively on PyFlink, but rather just occasionally. So instead of > > having a whole separate IDE and project for one module I'm trying to get > > the whole Flink project to work as one in IntelliJ. Do PyFlink developers > > generally work solely on PyFlink and nothing else / do you switch IDEs > for > > everything? > > > > > > Best > > Ingo > > > > On Wed, Aug 18, 2021 at 1:20 PM Dian Fu <dia...@apache.org> wrote: > > > > > Hi Ingo, > > > > > > Thanks a lot for starting up this discussion. I have not tried IntelliJ > > and > > > I’m using PyCharm for PyFlink development. Have you tried this > guideline? > > > [1] > > > It’s written in 1.9 and I think it should still work. > > > > > > Regards, > > > Dian > > > > > > [1] > > > > > > > > > https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/flinkdev/ide_setup/#pycharm > > > > > > On Wed, Aug 18, 2021 at 5:38 PM Till Rohrmann <trohrm...@apache.org> > > > wrote: > > > > > > > Thanks for starting this discussion Ingo. I guess that Dian can > > probably > > > > answer this question. I am big +1 for updating our documentation for > > how > > > to > > > > set up the IDE since this problem will probably be encountered a > couple > > > of > > > > times. > > > > > > > > Cheers, > > > > Till > > > > > > > > On Wed, Aug 18, 2021 at 11:03 AM Ingo Bürk <i...@ververica.com> > wrote: > > > > > > > > > Hello @dev, > > > > > > > > > > like probably most of you, I am using IntelliJ to work on Flink. > > > Lately I > > > > > needed to also work with flink-python, and thus was wondering about > > how > > > > to > > > > > properly set it up to work in IntelliJ. So far, I have done the > > > > following: > > > > > > > > > > 1. Install the Python plugin > > > > > 2. Set up a custom Python SDK using a virtualenv > > > > > 3. Configure the flink-python module to use this Python SDK (rather > > > than > > > > a > > > > > Java SDK) > > > > > 4. Install as many of the Python dependencies as possible when > > IntelliJ > > > > > prompted me to do so > > > > > > > > > > This got me into a mostly working state, for example I can run > tests > > > from > > > > > the IDE, at least the ones I tried so far. However, there are two > > > > concerns: > > > > > > > > > > a) Step (3) will be undone on every Maven import since Maven sets > the > > > SDK > > > > > for the module > > > > > b) Step (4) installed most, but a couple of Python dependencies > could > > > not > > > > > be installed, though so far that didn't cause noticeable problems > > > > > > > > > > I'm wondering if there are additional / different steps to do, > > > > specifically > > > > > for (a), and maybe how the PyFlink developers are configuring this > in > > > > their > > > > > IDE. The IDE setup guide didn't seem to contain information about > > that > > > > > (only about separately setting up this module in PyCharm). Ideally, > > we > > > > > could even update the IDE setup guide. > > > > > > > > > > > > > > > Best > > > > > Ingo > > > > > > > > > > > > > > >
