Thanks Dian Fu, On 1) This is more in reference to how if we modify things like the builder pattern, we could end up changing certain configuration patterns. I will reframe this to say - Currently there are no expected interfaces that will be removed, but as the work evolves there may be some required changes to determined non-pythonic areas, but the best effort will be made to mirror or maintain an escape hatch.
2) I understand the desire to limit the scope here because we don't want to go down the Pandas parity rabbit hole. Would it suffice to limit the scope to foundational operations (Creation, Inspection and I/O) and Core manipulation (Selection, Indexing and Filtering). These could be further outlined in issues. Maybe the following would suffice for this FLIP: Dataframe Methods Add friendly dataframe methods for creation, inspection and I/O that exist in other data libraries like .read_json(), read_csv(), .head(), .show() and .display(). Reference table columns as attributes Allow pandas-like table.<my-col> reference in addition to col(“<my-col>”) for all table API arguments Kwargs aliasing Allow polars-like table.agg(a_sum=<expr>) in addition to table.select(<expr>.alias(“a_sum”) for providing named aliases via kwargs 3) I am ok with removing this for now although I do wish there was an easier way to include some of the most common connector interfaces. - Zander On Sun, Jun 29, 2025 at 11:13 PM Dian Fu <dian0511...@gmail.com> wrote: > Hi Zander, > > Thanks for driving this effort! Big +1 overally. This will be a good > improvement for Python users. > > Some quick questions about this FLIP: > > 1) Low-Level Knobs: Certain low-level, non-Pythonic configuration > options may be deprecated or hidden to simplify the API surface. > > Could you give some examples on which configuration options do you mean? > > 2) Introduction of user-friendly methods on the Table object for data > preview, such as .show() and .display(), similar to those in other > data-frame libraries. > > Dataframe style APIs are widely adopted in the Python world. However, > there are many convenient APIs in the DataFrame, I guess it deserves a > separate FLIP to discuss which kinds of API we want to borrow from it. > > 3) Package top connectors (Kafka, Parquet, S3) to reduce friction from > manual JAR downloads. > > I'm not sure if this is feasible since the connector implementations > have been moved to separate repos. However, I agree that the > experience should be improved. Maybe we could provide some guides to > improve the experience. > > Regards, > Dian > > > > > On Sat, Jun 28, 2025 at 5:24 AM Alexander Matheson > <a.w.mathe...@gmail.com> wrote: > > > > Hi devs, > > > > I would like to start a discussion about a new FLIP for a rather large > > umbrella of work concerning PyFlink that Dian Fu, Xingbo Huang, myself > and > > others have been coordinating around. > > > > As PyFlink continues to grow in adoption (downloads are up 10x YoY on > > PyPI!!!) it is overdue for additional investment to bring it inline with > > the expectations of the Python community. Given the increase in AI > > workloads shifting to real-time, these improvements will also help to > > support the net-new Flink users coming from that space. > > > > The Project is called The Zen of Flink as an ode to the driving > principles > > of Python called the Zen of Python and is broadly about making Flink more > > Pythonic. The work falls into six categories across API design, > > documentation, debuggability, local development, integration with the > > ecosystem and general usability. Not all of the work is concretely scoped > > yet and is not planned to be as more improvements will arise as we work > on > > this effort. > > > > The details of the FLIP can be found in the google doc linked below. > > > > > https://docs.google.com/document/d/18_u1XA9C_zdY_fu1OtQDwYyIk_TwjUfzAUOzhGxbN6w/edit?usp=sharing > > > > Looking forward to the discussion. > > > > Best, > > > > Zander >
