makarandhinge opened a new issue, #747:
URL: https://github.com/apache/wayang/issues/747
While working on a pipeline like PostgresTableSource → Join → SqlToStream →
Java map → collect, I faced a runtime ClassCastException. The issue is
happening because after the SQL join, the data is actually in a tuple-like
structure (like Tuple2<Record, Record>), but during the SqlToStream conversion,
this structure is not preserved properly and everything is treated as a Record.
Because of this, when I try to use the data in the Java map step (for
example accessing fields or casting types), it assumes a wrong structure and
crashes at runtime (e.g., String cannot be cast to Integer). There is no
validation or check between SQL and Java stages, so the mismatch is not caught
early and only fails during execution.
Ideally, the correct schema/type should be preserved (or validated) when
moving from SQL to Java, so that such issues don’t happen at runtime.
for example -
CREATE TABLE table_a (
id INT,
name TEXT
);
CREATE TABLE table_b (
id TEXT, -- different type
value TEXT
);
<img width="1806" height="718" alt="Image"
src="https://github.com/user-attachments/assets/11d0e4c4-c7a2-4eb1-befb-306fc9eb1d4e";
/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
