goutamadwant opened a new pull request, #13141:
URL: https://github.com/apache/seatunnel/pull/13141

   ### Purpose of this pull request
   
   Closes #13140. Related: #10967, #10968.
   
   Adds `catalog_type = rest` for the Paimon source, sink, catalog factory and 
SaveMode path. REST reuses `catalog_uri`, takes `warehouse` as the managed 
warehouse name, and forwards Paimon 1.1.1's `token.provider`, `token`, 
`dlf.access-key-id`, `dlf.access-key-secret`, `dlf.security-token` and 
`dlf.region`. No dependency upgrade.
   
   Only explicitly configured Hadoop settings are passed to the REST catalog, 
so Hadoop's built-in defaults cannot override server-supplied defaults; REST 
option keys inside `paimon.hadoop.conf` or XML files are rejected. 
`catalog_uri` must be HTTP or HTTPS with a host and no user information (plain 
HTTP outside loopback logs a warning), `token.provider` must be `bear` or `dlf` 
with the matching credentials present, and REST catalogs are closed when no 
longer used. DLF credentials are masked in the parsed-config log; encryption 
behaviour is unchanged. `filesystem` and `hive` are unchanged.
   
   ### Does this PR introduce _any_ user-facing change?
   
   Yes: a new `catalog_type` value and the authentication options above, 
documented in the EN and ZH source and sink guides, including what is not 
supported (DLF token loaders and refresh) and two dependency limits (Paimon's 
`HadoopUtils` DEBUG logging can print storage settings; Paimon's HTTP client 
forwards `x-dlf-security-token` across host redirects). Existing configurations 
are unaffected.
   
   ### How was this patch tested?
   
   - Unit tests fail before the change and pass after: REST option forwarding, 
explicit-only storage options, reserved-key rejection, URI and credential 
validation, catalog closure, and unchanged filesystem/Hive option maps. 129 
Paimon unit tests and `ConfigBuilderTest` pass on JDK 11 and 17.
   - `PaimonRestCatalogIT` (3 tests) against Paimon's REST test server on JDK 
11 and 17: SaveMode table creation, write and commit, read-back through the 
source, discovery, wrong-token rejection, DLF request signing, and 
server-supplied storage defaults.
   - Not covered: live DLF, remote storage authorization, credential refresh, 
Flink and Spark engines.
   
   ### Check list
   
   * [x] If any new Jar binary package adding in your PR, please add License 
Notice according to the [New License 
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
 (test-scoped only; no new distributed dependency)
   * [x] Update documentation to describe the new feature
   * [ ] Update `incompatible-changes.md` if necessary (not needed: additive)
   * [x] Check the connector integration files (plugin mapping, dist pom, label 
scope and plugin config unchanged; the REST IT joins the existing Paimon e2e 
module)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to