gomnitrix commented on code in PR #3045:
URL: https://github.com/apache/iggy/pull/3045#discussion_r3035541908
##########
core/connectors/sinks/iceberg_sink/src/props.rs:
##########
@@ -30,14 +30,49 @@ pub fn init_props(config: &IcebergSinkConfig) ->
Result<HashMap<String, String>,
fn get_props_s3(config: &IcebergSinkConfig) -> Result<HashMap<String, String>,
Error> {
let mut props: HashMap<String, String> = HashMap::new();
props.insert("s3.region".to_string(), config.store_region.clone());
- props.insert(
- "s3.access-key-id".to_string(),
- config.store_access_key_id.clone(),
- );
- props.insert(
- "s3.secret-access-key".to_string(),
- config.store_secret_access_key.clone(),
- );
+ if let Some(access_key_id) = &config.store_access_key_id {
Review Comment:
Thanks for the docker-compose snippet! Here is how I verified it:
1. Automated CI Test: In this PR, I added a new integration test
(iceberg_sink_uses_default_credential_chain) specifically to cover this
fallback scenario. It has passed in the latest CI run: `PASS [ 18.521s] (
770/1037) integration::mod
connectors::iceberg::iceberg_sink::iceberg_sink_uses_default_credential_chain`
2. Manual E2E Testing: Following your suggestion, I also set up the docker
environment locally and performed a manual test. Passing the credentials purely
via env vars triggered the fallback mechanism correctly, and the messages were
routed successfully:
```
# 1. The connector initializes and successfully falls back to default
credentials:
2026-04-04T11:28:28.680638Z INFO connector:
connector_target="iggy_connector_iceberg_sink::sink" Opened Iceberg sink
connector with ID: 1 for URL: http://rest:8181. No explicit credentials
provided, falling back to default credential provider chain
2026-04-04T11:28:28.680661Z INFO connector:
connector_target="iggy_connector_iceberg_sink::sink" Configuring Iceberg
catalog with the following config:
-region: us-east-1
-url: http://minio:9000
-store class: S3
-catalog type: REST
2026-04-04T11:28:28.692411Z INFO connector:
connector_target="iggy_connector_iceberg_sink::router::static_router" Static
router found 1 tables on iceberg catalog from 1 tables declared
2026-04-04T11:28:28.692612Z INFO iggy_connectors::sink: Sink container with
name: Iceberg sink (iceberg), initialized successfully with ID: 1.
# 2. Sending a test message via CLI to the running server:
$ cargo run --bin iggy -- -u iggy -p password message send test_stream
test_topic "{\"id\": 1, \"name\": \"hello_iggy\"}"
Executing send messages to topic with ID: test_topic and stream with ID:
test_stream
Sent messages to topic with ID: test_topic and stream with ID: test_stream
# 3. The connector immediately processes and routes it to Iceberg:
2026-04-04T11:29:54.643126Z INFO connector:
connector_target="iggy_connector_iceberg_sink::router::static_router" Routed 1
messages to iceberg table messages successfully
```
I've also added the result into the PR details, hope this looks good to you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
