This is an automated email from the ASF dual-hosted git repository.
numinnex pushed a change to branch replica_bootstrap
in repository https://gitbox.apache.org/repos/asf/iggy.git
omit 39ce6cbf3 fix cargo machete & sort
omit 7fabdb0ca finito
omit f1279e46c tmp v2
omit fbafdee10 temp
add 0b290b00c ci(csharp): auto-retry flaky integration tests (#3160)
add 037edec4c chore(deps): bump ctor, cucumber, libc, rustls + security
patches (#3161)
add aa4762fd5 docs: redesign header with theme-aware logo and SDK badges
(#3162)
add 868ae6296 docs: Add trendshift's github ranking badge to README (#3170)
add 7de9d093f chore(deps): bump python-dotenv from 1.2.1 to 1.2.2 in
/foreign/python (#3150)
add e1b1b22a7 fix(go,csharp): remove duplicate consumer groups in
DeserializeClient/MapClient (#3164)
add 499a26d88 feat(connectors): implement Delta Lake Sink Connector (#2889)
add 48a297dfa chore(web): update dependencies (#3185)
add cca0e04e7 feat(configs): add ServerNgConfig schema with message_bus
section (#3189)
add eb20ac5ae feat(java): Implement CLI and resource provisioner for
pinned producer benchmark (#3159)
add 57646b233 chore(deps): bump mako from 1.3.10 to 1.3.11 in /bdd/python
(#3137)
add 9a9254a3a ci: bump GitHub Actions to Node.js 24-compatible versions
(#3086)
add 611fca04e fix(ci): honor staged file scope in lint hooks (#3188)
add 842538572 test(csharp): add leader_redirection scenario to BDD tests
(#3193)
add 4f55e9563 feat(connectors): implement Avro payload support with
separate encoder, decoder and transform crates (#3141)
add 87e22a1cc feat(go): add delete segments support (#3191)
add 5fbeddca4 fix(connectors): reject duplicate iggy_sink_open and
iggy_source_open (#3179)
add 55705f395 chore(repo): add ASF incubator status and update readme
(#3208)
add 93227c17d test(server): add QUIC transport coverage to consumer group
scenarios (#3207)
add f02a43a8d ci: allow node and common scopes in PR titles (#3209)
add e93785249 feat(message_bus): add QUIC, TCP-TLS, WS, WSS transports for
SDK clients (#3192)
add a0e286050 temp
add df1cd74f6 tmp v2
add dada05260 finito
add 06ee92310 fix cargo machete & sort
add a1d20d79b server-ng config
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (39ce6cbf3)
\
N -- N -- N refs/heads/replica_bootstrap (a1d20d79b)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../actions/csharp-dotnet/post-merge/action.yml | 2 +-
.github/actions/csharp-dotnet/pre-merge/action.yml | 6 +-
.github/actions/go/pre-merge/action.yml | 6 +-
.github/actions/node-npm/post-merge/action.yml | 2 +-
.github/actions/node-npm/pre-merge/action.yml | 2 +-
.../actions/python-maturin/post-merge/action.yml | 4 +-
.../actions/python-maturin/pre-merge/action.yml | 8 +-
.github/actions/utils/docker-buildx/action.yml | 12 +-
.../actions/utils/setup-cpp-with-cache/action.yml | 2 +-
.../actions/utils/setup-go-with-cache/action.yml | 6 +-
.../actions/utils/setup-java-with-cache/action.yml | 4 +-
.../actions/utils/setup-node-with-cache/action.yml | 4 +-
.github/workflows/_build_python_wheels.yml | 26 +-
.github/workflows/_build_rust_artifacts.yml | 16 +-
.github/workflows/_common.yml | 32 +-
.github/workflows/_detect.yml | 4 +-
.github/workflows/_publish_rust_crates.yml | 2 +-
.github/workflows/_test.yml | 18 +-
.github/workflows/_test_bdd.yml | 4 +-
.github/workflows/_test_examples.yml | 10 +-
.github/workflows/coverage-baseline.yml | 34 +-
.github/workflows/edge-release.yml | 14 +-
.github/workflows/post-merge.yml | 2 +-
.github/workflows/pre-merge.yml | 2 +-
.github/workflows/publish.yml | 26 +-
.pre-commit-config.yaml | 26 +-
Cargo.lock | 1665 +++++++++++++++++---
Cargo.toml | 21 +-
DEPENDENCIES.md | 122 +-
Dockerfile | 12 +
README.md | 53 +-
bdd/Dockerfile | 12 +
bdd/csharp/Dockerfile | 12 +
bdd/docker-compose.yml | 9 +-
bdd/go/Dockerfile | 12 +
...eature_delete.go => segments_feature_delete.go} | 35 +-
bdd/java/Dockerfile | 12 +
bdd/node/Dockerfile | 12 +
bdd/python/Dockerfile | 11 +
bdd/python/uv.lock | 6 +-
bdd/rust/Dockerfile | 12 +
core/ai/mcp/Dockerfile | 12 +
core/bench/dashboard/server/Dockerfile | 12 +
core/binary_protocol/Cargo.toml | 2 +-
core/binary_protocol/README.md | 29 +
core/binary_protocol/src/consensus/header.rs | 29 +
core/binary_protocol/src/consensus/iobuf.rs | 6 +
core/binary_protocol/src/consensus/mod.rs | 3 +-
core/binary_protocol/src/lib.rs | 4 +-
core/cli/README.md | 38 +-
core/common/Cargo.toml | 2 +-
core/common/README.md | 29 +
core/configs/src/lib.rs | 4 +
core/configs/src/server_config/defaults.rs | 3 +-
core/configs/src/server_config/displays.rs | 24 +-
core/configs/src/server_config/server.rs | 30 +-
core/configs/src/server_config/validators.rs | 27 +-
core/configs/src/server_ng_config/defaults.rs | 221 +++
core/configs/src/server_ng_config/displays.rs | 167 ++
core/configs/src/server_ng_config/message_bus.rs | 357 +++++
.../src/{server_config => server_ng_config}/mod.rs | 24 +-
core/configs/src/server_ng_config/quic.rs | 223 +++
core/configs/src/server_ng_config/server_ng.rs | 224 +++
.../src/{server_config => server_ng_config}/tcp.rs | 7 +
core/configs/src/server_ng_config/validators.rs | 158 ++
.../websocket.rs | 10 +
core/connectors/runtime/Dockerfile | 12 +
.../example_config/connectors/delta_sink.toml} | 14 +-
core/connectors/runtime/src/configs/connectors.rs | 21 +-
core/connectors/runtime/src/sink.rs | 28 +-
core/connectors/runtime/src/source.rs | 21 +-
core/connectors/sdk/Cargo.toml | 1 +
core/connectors/sdk/src/convert.rs | 114 ++
core/connectors/sdk/src/decoders/avro.rs | 442 ++++++
core/connectors/sdk/src/decoders/mod.rs | 1 +
core/connectors/sdk/src/encoders/avro.rs | 604 +++++++
core/connectors/sdk/src/encoders/flatbuffer.rs | 1 +
core/connectors/sdk/src/encoders/mod.rs | 1 +
core/connectors/sdk/src/encoders/proto.rs | 12 +
core/connectors/sdk/src/lib.rs | 17 +-
core/connectors/sdk/src/sink.rs | 7 +
core/connectors/sdk/src/source.rs | 7 +
core/connectors/sdk/src/transforms/avro_convert.rs | 398 +++++
core/connectors/sdk/src/transforms/mod.rs | 8 +
.../connectors/sdk/src/transforms/proto_convert.rs | 17 +
.../sinks/{stdout_sink => delta_sink}/Cargo.toml | 11 +-
core/connectors/sinks/delta_sink/README.md | 92 ++
core/connectors/sinks/delta_sink/config.toml | 62 +
core/connectors/sinks/delta_sink/src/coercions.rs | 680 ++++++++
core/connectors/sinks/delta_sink/src/lib.rs | 97 ++
core/connectors/sinks/delta_sink/src/sink.rs | 181 +++
core/connectors/sinks/delta_sink/src/storage.rs | 301 ++++
core/connectors/sinks/http_sink/src/lib.rs | 8 +
core/integration/Cargo.toml | 2 +
.../{stdout/stdout_sink.rs => delta/delta_sink.rs} | 120 +-
.../tests/connectors/{api => delta}/mod.rs | 2 +-
.../tests/connectors/{http => delta}/sink.toml | 2 +-
.../tests/connectors/fixtures/delta/fixture.rs | 391 +++++
.../connectors/{http => fixtures/delta}/mod.rs | 4 +-
core/integration/tests/connectors/fixtures/mod.rs | 2 +
core/integration/tests/connectors/mod.rs | 1 +
core/integration/tests/server/cg.rs | 13 +-
core/message_bus/Cargo.toml | 12 +
core/message_bus/src/client_listener/mod.rs | 103 ++
core/message_bus/src/client_listener/quic.rs | 158 ++
.../{client_listener.rs => client_listener/tcp.rs} | 18 +-
core/message_bus/src/client_listener/tcp_tls.rs | 140 ++
.../{client_listener.rs => client_listener/ws.rs} | 68 +-
core/message_bus/src/client_listener/wss.rs | 131 ++
core/message_bus/src/config.rs | 320 +++-
core/message_bus/src/connector.rs | 51 +-
core/message_bus/src/error.rs | 1 +
core/message_bus/src/framing.rs | 78 +-
core/message_bus/src/installer.rs | 542 -------
core/message_bus/src/installer/common.rs | 57 +
core/message_bus/src/installer/conn_info.rs | 119 ++
core/message_bus/src/installer/mod.rs | 151 ++
core/message_bus/src/installer/quic.rs | 64 +
core/message_bus/src/installer/replica.rs | 337 ++++
core/message_bus/src/installer/tcp.rs | 287 ++++
core/message_bus/src/installer/tcp_tls.rs | 76 +
core/message_bus/src/installer/ws.rs | 56 +
core/message_bus/src/installer/wss.rs | 74 +
core/message_bus/src/lib.rs | 422 ++++-
.../src/lifecycle/connection_registry.rs | 148 +-
core/message_bus/src/lifecycle/mod.rs | 4 +-
core/message_bus/src/lifecycle/shutdown.rs | 96 +-
core/message_bus/src/replica/io.rs | 514 ++++++
core/message_bus/src/replica/listener.rs | 250 +++
.../message_bus/src/{transports => replica}/mod.rs | 13 +-
core/message_bus/src/replica_io.rs | 154 --
core/message_bus/src/replica_listener.rs | 164 --
core/message_bus/src/socket_opts.rs | 43 +-
core/message_bus/src/transports/mod.rs | 175 +-
core/message_bus/src/transports/quic.rs | 727 +++++++++
core/message_bus/src/transports/tcp.rs | 431 +++++
core/message_bus/src/transports/tcp_tls.rs | 914 +++++++++++
core/message_bus/src/transports/tls/mod.rs | 257 +++
core/message_bus/src/transports/ws.rs | 524 ++++++
core/message_bus/src/transports/wss.rs | 697 ++++++++
core/message_bus/src/writer_task.rs | 117 --
core/message_bus/tests/backpressure.rs | 3 +-
core/message_bus/tests/cancel_unsafe.rs | 140 ++
core/message_bus/tests/common/mod.rs | 107 +-
core/message_bus/tests/connection_lost_notify.rs | 3 +-
core/message_bus/tests/directional_connection.rs | 4 +-
core/message_bus/tests/duplicate_client_id.rs | 50 +-
core/message_bus/tests/graceful_shutdown.rs | 3 +-
core/message_bus/tests/head_of_line.rs | 3 +-
core/message_bus/tests/installer_panic_cleanup.rs | 348 ++++
core/message_bus/tests/quic_client_roundtrip.rs | 230 +++
core/message_bus/tests/reconnect.rs | 3 +-
core/message_bus/tests/reconnect_skip_connected.rs | 3 +-
core/message_bus/tests/replica_roundtrip.rs | 3 +-
core/message_bus/tests/shard_zero_gating.rs | 218 ++-
core/message_bus/tests/tcp_client_roundtrip.rs | 3 +-
core/message_bus/tests/tcp_tls_cancel_safe.rs | 167 ++
core/message_bus/tests/tcp_tls_client_listener.rs | 193 +++
core/message_bus/tests/tcp_tls_client_roundtrip.rs | 160 ++
core/message_bus/tests/vectored_batch.rs | 3 +-
core/message_bus/tests/ws_client_roundtrip.rs | 144 ++
core/message_bus/tests/wss_client_listener.rs | 185 +++
core/message_bus/tests/wss_client_roundtrip.rs | 161 ++
core/sdk/Cargo.toml | 3 +-
core/sdk/README.md | 154 ++
core/server-ng/Dockerfile | 12 +
core/server-ng/LISTENER_SUPPORT_PLAN.md | 286 ++++
core/server-ng/config.toml | 110 +-
core/server-ng/src/bootstrap.rs | 62 +-
core/server-ng/src/config_writer.rs | 4 +-
core/server-ng/src/session_manager.rs | 9 +-
core/server/Dockerfile | 12 +
core/shard/src/builder.rs | 7 +-
core/shard/src/coordinator.rs | 318 +++-
core/shard/src/lib.rs | 27 +-
core/shard/src/router.rs | 22 +-
examples/rust/src/sink-data-producer/main.rs | 3 +
foreign/cpp/README.md | 16 +
foreign/csharp/Directory.Packages.props | 1 +
.../Iggy_SDK.Tests.BDD/Context/TestContext.cs | 5 +
.../csharp/Iggy_SDK.Tests.BDD/Context/TestHooks.cs | 19 +-
.../Iggy_SDK.Tests.BDD/Iggy_SDK.Tests.BDD.csproj | 9 +-
.../StepDefinitions/LeaderRedirectionSteps.cs | 317 ++++
.../Fixtures/IggyClusterFixture.cs | 70 +-
.../Iggy_SDK.Tests.Integration.csproj | 1 +
foreign/csharp/Iggy_SDK/Mappers/BinaryMapper.cs | 32 +-
foreign/csharp/README.md | 14 +
foreign/go/README.md | 14 +
.../binary_response_deserializer.go | 28 +-
...{tcp_utilities.go => tcp_segment_management.go} | 26 +-
foreign/go/contracts/client.go | 9 +
foreign/go/internal/command/code.go | 1 +
.../command/segment.go} | 29 +-
.../command/{topic_test.go => segment_test.go} | 35 +-
foreign/java/README.md | 24 +-
foreign/java/{buildSrc => bench}/build.gradle.kts | 21 +-
.../java/org/apache/iggy/bench/IggyBench.java} | 21 +-
.../tcp/async/TcpAsyncPinnedProducer.java | 51 +
.../apache/iggy/bench/cli/IggyBenchCommand.java | 162 ++
.../iggy/bench/cli/PinnedProducerCommand.java | 111 ++
.../iggy/bench/exception/BenchmarkException.java} | 19 +-
.../iggy/bench/models/cli/GlobalCliArgs.java | 85 +
.../bench/models/cli/PinnedProducerCliArgs.java | 43 +
.../models/provision/ProvisionedResources.java} | 4 +-
.../iggy/bench/provision/ResourceProvisioner.java | 100 ++
.../docker/Dockerfile.flink2_10-java21 | 29 +
foreign/java/gradle/libs.versions.toml | 2 +
foreign/java/settings.gradle.kts | 3 +
foreign/node/README.md | 14 +
foreign/python/.devcontainer/Dockerfile | 12 +
foreign/python/Dockerfile.test | 11 +
foreign/python/README.md | 14 +
foreign/python/uv.lock | 6 +-
scripts/ci/markdownlint.sh | 70 +-
scripts/ci/shellcheck.sh | 125 +-
scripts/ci/trailing-whitespace.sh | 5 +-
scripts/ci/uv-lock-check.sh | 7 +
scripts/extract-version.sh | 2 +
web/Dockerfile | 12 +
web/package-lock.json | 1314 ++++++++++-----
web/package.json | 40 +-
221 files changed, 19573 insertions(+), 2636 deletions(-)
copy bdd/go/tests/tcp_test/{partitions_feature_delete.go =>
segments_feature_delete.go} (75%)
create mode 100644 core/binary_protocol/README.md
create mode 100644 core/common/README.md
create mode 100644 core/configs/src/server_ng_config/defaults.rs
create mode 100644 core/configs/src/server_ng_config/displays.rs
create mode 100644 core/configs/src/server_ng_config/message_bus.rs
copy core/configs/src/{server_config => server_ng_config}/mod.rs (54%)
create mode 100644 core/configs/src/server_ng_config/quic.rs
create mode 100644 core/configs/src/server_ng_config/server_ng.rs
copy core/configs/src/{server_config => server_ng_config}/tcp.rs (85%)
create mode 100644 core/configs/src/server_ng_config/validators.rs
copy core/configs/src/{server_config => server_ng_config}/websocket.rs (88%)
copy core/connectors/{sinks/stdout_sink/config.toml =>
runtime/example_config/connectors/delta_sink.toml} (79%)
create mode 100644 core/connectors/sdk/src/decoders/avro.rs
create mode 100644 core/connectors/sdk/src/encoders/avro.rs
create mode 100644 core/connectors/sdk/src/transforms/avro_convert.rs
copy core/connectors/sinks/{stdout_sink => delta_sink}/Cargo.toml (84%)
create mode 100644 core/connectors/sinks/delta_sink/README.md
create mode 100644 core/connectors/sinks/delta_sink/config.toml
create mode 100644 core/connectors/sinks/delta_sink/src/coercions.rs
create mode 100644 core/connectors/sinks/delta_sink/src/lib.rs
create mode 100644 core/connectors/sinks/delta_sink/src/sink.rs
create mode 100644 core/connectors/sinks/delta_sink/src/storage.rs
copy core/integration/tests/connectors/{stdout/stdout_sink.rs =>
delta/delta_sink.rs} (71%)
copy core/integration/tests/connectors/{api => delta}/mod.rs (98%)
copy core/integration/tests/connectors/{http => delta}/sink.toml (94%)
create mode 100644 core/integration/tests/connectors/fixtures/delta/fixture.rs
copy core/integration/tests/connectors/{http => fixtures/delta}/mod.rs (92%)
create mode 100644 core/message_bus/src/client_listener/mod.rs
create mode 100644 core/message_bus/src/client_listener/quic.rs
copy core/message_bus/src/{client_listener.rs => client_listener/tcp.rs} (84%)
create mode 100644 core/message_bus/src/client_listener/tcp_tls.rs
rename core/message_bus/src/{client_listener.rs => client_listener/ws.rs} (51%)
create mode 100644 core/message_bus/src/client_listener/wss.rs
delete mode 100644 core/message_bus/src/installer.rs
create mode 100644 core/message_bus/src/installer/common.rs
create mode 100644 core/message_bus/src/installer/conn_info.rs
create mode 100644 core/message_bus/src/installer/mod.rs
create mode 100644 core/message_bus/src/installer/quic.rs
create mode 100644 core/message_bus/src/installer/replica.rs
create mode 100644 core/message_bus/src/installer/tcp.rs
create mode 100644 core/message_bus/src/installer/tcp_tls.rs
create mode 100644 core/message_bus/src/installer/ws.rs
create mode 100644 core/message_bus/src/installer/wss.rs
create mode 100644 core/message_bus/src/replica/io.rs
create mode 100644 core/message_bus/src/replica/listener.rs
copy core/message_bus/src/{transports => replica}/mod.rs (69%)
delete mode 100644 core/message_bus/src/replica_io.rs
delete mode 100644 core/message_bus/src/replica_listener.rs
create mode 100644 core/message_bus/src/transports/quic.rs
create mode 100644 core/message_bus/src/transports/tcp.rs
create mode 100644 core/message_bus/src/transports/tcp_tls.rs
create mode 100644 core/message_bus/src/transports/tls/mod.rs
create mode 100644 core/message_bus/src/transports/ws.rs
create mode 100644 core/message_bus/src/transports/wss.rs
delete mode 100644 core/message_bus/src/writer_task.rs
create mode 100644 core/message_bus/tests/cancel_unsafe.rs
create mode 100644 core/message_bus/tests/installer_panic_cleanup.rs
create mode 100644 core/message_bus/tests/quic_client_roundtrip.rs
create mode 100644 core/message_bus/tests/tcp_tls_cancel_safe.rs
create mode 100644 core/message_bus/tests/tcp_tls_client_listener.rs
create mode 100644 core/message_bus/tests/tcp_tls_client_roundtrip.rs
create mode 100644 core/message_bus/tests/ws_client_roundtrip.rs
create mode 100644 core/message_bus/tests/wss_client_listener.rs
create mode 100644 core/message_bus/tests/wss_client_roundtrip.rs
create mode 100644 core/sdk/README.md
create mode 100644 core/server-ng/LISTENER_SUPPORT_PLAN.md
create mode 100644
foreign/csharp/Iggy_SDK.Tests.BDD/StepDefinitions/LeaderRedirectionSteps.cs
copy foreign/go/client/tcp/{tcp_utilities.go => tcp_segment_management.go}
(75%)
copy foreign/go/{client/tcp/tcp_clients_management.go =>
internal/command/segment.go} (59%)
copy foreign/go/internal/command/{topic_test.go => segment_test.go} (58%)
copy foreign/java/{buildSrc => bench}/build.gradle.kts (58%)
copy
foreign/java/{java-sdk/src/main/java/org/apache/iggy/message/PollingKind.java
=> bench/src/main/java/org/apache/iggy/bench/IggyBench.java} (71%)
create mode 100644
foreign/java/bench/src/main/java/org/apache/iggy/bench/benchmarks/tcp/async/TcpAsyncPinnedProducer.java
create mode 100644
foreign/java/bench/src/main/java/org/apache/iggy/bench/cli/IggyBenchCommand.java
create mode 100644
foreign/java/bench/src/main/java/org/apache/iggy/bench/cli/PinnedProducerCommand.java
copy
foreign/java/{java-sdk/src/main/java/org/apache/iggy/message/PollingKind.java
=> bench/src/main/java/org/apache/iggy/bench/exception/BenchmarkException.java}
(75%)
create mode 100644
foreign/java/bench/src/main/java/org/apache/iggy/bench/models/cli/GlobalCliArgs.java
create mode 100644
foreign/java/bench/src/main/java/org/apache/iggy/bench/models/cli/PinnedProducerCliArgs.java
copy
foreign/java/{java-sdk/src/main/java/org/apache/iggy/consumergroup/ConsumerGroupMember.java
=>
bench/src/main/java/org/apache/iggy/bench/models/provision/ProvisionedResources.java}
(85%)
create mode 100644
foreign/java/bench/src/main/java/org/apache/iggy/bench/provision/ResourceProvisioner.java
