Hi community, I am exploring PyFlink and I wonder if it is possible to use PyFlink together with all these ML libs that ML engineers normally use: PyTorch, Tensorflow, Scikit Learn, Xgboost, LightGBM, etc.
According to this SO thread <https://stackoverflow.com/questions/38187637/integrating-scikit-learn-with-pyspark>, PySpark cannot use Scikit Learn directly inside UDF because Scikit Learn algorithms are not implemented to be distributed, while Spark runs distributedly. Given PyFlink is similar to PySpark, I guess the answer may be "no". But I would love to double check, and to see what I need to do to make PyFlink able to define UDFs using these ML libs. (This question is cross-posted on StackOverflow https://stackoverflow.com/questions/66631859/can-i-use-pyflink-together-with-pytorch-tensorflow-scikitlearn-xgboost-lightgbm ) Thanks. Best, Yik San
