goutamadwant opened a new pull request, #13094:
URL: https://github.com/apache/seatunnel/pull/13094
### Purpose of this pull request
Related to #10753.
This contribution to Apache SeaTunnel adds a bounded Mailchimp
audience-member source using the Marketing API, reusing the existing HTTP and
JSON infrastructure.
- Support API-key authentication, data-center endpoint selection and bounded
count/offset pagination.
- Preserve decimal precision during schema conversion and bound response
sizes.
- Retry HTTP 429 and server errors at the same offset with finite budgets;
retain connection reuse for successful pages.
- Keep network operations and completion callbacks outside checkpoint locks,
with cancellation and incomplete-response cleanup.
- Add plugin registration, distribution configuration, English and Chinese
documentation, and Zeta E2E coverage.
### Does this PR introduce _any_ user-facing change?
Yes. Users can configure `Mailchimp` as a bounded batch source with an API
key in `token`, an audience `list_id` and a schema.
The first implementation uses one reader and replays the batch after
recovery. It does not provide deletion CDC, exactly-once delivery or a
transactional snapshot. Concurrent audience changes can affect offset
pagination; this limitation is documented. Existing connectors are unchanged.
### How was this patch tested?
- Reproduced missing factory discovery before implementation; the unchanged
probe discovers the new factory afterward.
- Passed 75 focused tests on JDK 11 and 75 on JDK 17, covering
configuration, pagination, decimal precision, invalid responses, size limits,
retries, connection reuse, cancellation, cleanup and completion locking.
- Passed Zeta Docker E2E with engine retries disabled: empty HTTP 429 and
503 responses followed by two member pages, exactly four authenticated
requests, three output rows and terminal FINISHED status.
- Passed adjacent HTTP-base tests on JDK 11 and 17, plus focused Asana and
PostHog tests on JDK 11.
- Passed scoped builds, verification and `spotless:apply`. Self-review and
independent correctness/security reviews completed.
Reproduction commands:
- With each JDK selected through `JAVA_HOME`: `mvn -pl
:connector-http-mailchimp test`
- Build E2E dependencies: `mvn -Pci -pl :connector-mailchimp-e2e -am
-DskipUT=true -DskipIT=true -Dskip.ui=true install`
- Run integration test: `RUN_ENGINES=zeta mvn -Pci -pl
:connector-mailchimp-e2e -DskipUT=true -DskipIT=false -Dit.test=MailchimpIT
verify`
The E2E uses a deterministic HTTP fixture, not a live Mailchimp account.
Live-service acceptance, Flink/Spark E2E, crash/restart recovery and
full-repository verification were not run. Hosted CI is queued at submission.
### Check list
* [x] No new third-party binary dependencies; existing HTTP/JSON
dependencies are reused.
* [x] Updated English and Chinese documentation and connector changelogs.
* [x] No incompatible changes to existing behavior; no
`incompatible-changes.md` update required.
* [x] Updated connector integration files:
1. `plugin-mapping.properties`
2. `seatunnel-dist/pom.xml`
3. `.github/workflows/labeler/label-scope-conf.yml`
4. Connector E2E module and tests
5. `config/plugin_config`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]