Hi Expert,
Usually when we write Flink-SQL program, usually we need to use
multiple tables to get the final result, this is due to sometimes it is not
possible to implement complicated logic in one SQL, sometimes due to the
clarity of logic. For example:
create view A as
select * from source where xxx;
create view B as
select * from A where xxx;
create view C as
select * from B where xxx;
insert into sink
select * from C where xxx;
But when we write complicated logic, we may accomplish it step by step,
make sure that the first step is correct, then go on with the next step. In
batch program such as Hive or Spark, we usually write SQL like this, step by
step.
For example:
create view A as
select * from source where xxx;
I want to check if the content in A is correct, if it is correct I go
on to write another SQL. But I do not want to define a sink for each step,
because it is not worthy just create a sink for such a “debug” step.
So is there a solution or best practice for such a scenario? How do we
easily debug or verify the correctness of a Flink SQL program?
Best
Henry
