gaborgsomogyi opened a new pull request, #250:
URL: https://github.com/apache/flink-connector-kafka/pull/250
Summary
Adds a new built-in SQL table function savepoint_get_kafka_offsets that
allows users to inspect Kafka source offsets stored in a Flink savepoint
directly from SQL.
What it does:
- Introduces GetKafkaSourceOffsetsTableFunction, a TableFunction that reads
KafkaPartitionSplit state from a savepoint and emits rows with (topic,
partition, starting-offset, stopping-offset) for each partition.
- Registers the function via KafkaDynamicBuiltInFunctionDefinitionFactory
using the state.kafka factory identifier, which integrates with the Flink state
module (`LOAD MODULE` state).
- Wires up the factory through Java SPI
(`META-INF/services/DynamicBuiltInFunctionDefinitionFactory`).
- Adds flink-state-processor-api as a provided dependency to support
savepoint loading and operator state deserialization.
- Bumps the Flink version from `2.1.0` to `2.2.0`.
Example usage:
```
LOAD MODULE state;
SELECT * FROM savepoint_get_kafka_offsets('/path/to/savepoint',
'my-kafka-source-uid');
```
Test plan
- `SavepointKafkaOffsetsTableFunctionTest#testReadKafkaOffsets` — runs the
function in batch mode against a pre-captured savepoint, verifies that two
partitions (test1:0 at offset -2, test1:1 at offset 1) are returned correctly.
- Existing unit and integration tests pass without regression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
