atharvalade opened a new issue, #3173: URL: https://github.com/apache/iggy/issues/3173
The core row-processing loop (iterate columns, extract values, build JSON payload, construct `ProducedMessage`) is duplicated in three places within `postgres_source/src/lib.rs` (2,394 lines): 1. `process_rows` (sequential polling path) 2. Inline closure inside `poll_tables_parallel` (non-chunked parallel path) 3. `poll_table_chunked` (chunked parallel path) Every new feature (like `flat_json_output`) must be wired into all three. Every bug fix must be applied three times. During the benchmark, adding flat JSON output required editing three separate blocks of nearly identical code. **Fix**: Extract the row-to-message conversion into a single shared function that all three paths call. The function signature would be roughly: `fn rows_to_messages(rows, table_name, config_flags, ...) -> Result<Vec<ProducedMessage>, Error>` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
