Re: Debugging SQL queries

2022-03-14 Thread Caizhi Weng
Hi! My understanding is that Flink SQL will generate Java code equivalent and > submit the compiled jar to the cluster as a job, is that correct? > Almost correct. Flink SQL will generate Java code and compile them to Java classes. These compiled classes are stored in transactions (a type of data

Re: Debugging SQL queries

2022-03-13 Thread dz902
Thanks! I have a few follow up questions. I have searched but was unable to find where to set the log level just for "org.apache.flink.table.runtime.generated.CompileUtils", so I set "rootLogger.level = DEBUG" in "log4j-cli.properties" and got the logs, but was unable to find generated code in "fl

Re: Debugging SQL queries

2022-03-13 Thread Caizhi Weng
Sorry for misleading. I mean if you enable checkpointing then selected results are only visible after the checkpoint completes. If there is no checkpointing the results will be instantly visible, just as the document records. Caizhi Weng 于2022年3月14日周一 11:12写道: > Hi! > > I see. So you're running

Re: Debugging SQL queries

2022-03-13 Thread Caizhi Weng
Hi! I see. So you're running a streaming job. "select" in a streaming job will only produce visible data when you enable checkpointing (this is due to the exactly-once guarantee of Flink), see [1] for more detail. See [2] on how to enable checkpointing for Flink SQL. Generated code are also in th

Re: Debugging SQL queries

2022-03-13 Thread dz902
Hi Caizhi, Thanks for the quick reply. I was just running a simple "SELECT * FROM tbl_name" against a table with Kafka connector, but no data showed up and no errors. Where can I find the generated code if I'm using SQL client? Thanks! On Mon, Mar 14, 2022 at 10:58 AM Caizhi Weng wrote: > Hi

Re: Debugging SQL queries

2022-03-13 Thread Caizhi Weng
Hi! For stages and logs you can refer to the web UI. For generated code set logging level of org.apache.flink.table.runtime.generated.CompileUtils to debug. What query are you running? If possible can you share your SQL in the mailing list? dz902 于2022年3月14日周一 10:42写道: > Hi, > > I'm trying to

Debugging SQL queries

2022-03-13 Thread dz902
Hi, I'm trying to debug SQL queries, from SQL client or Zeppelin notebook (job submitted to remote cluster). I have a query not getting any data. How do I debug? Can I see the actual code generated from the SQL query? Or is it possible to show all the stages, actions or logs generated by the quer