This is an automated email from the ASF dual-hosted git repository.
maciej pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/iggy.git
The following commit(s) were added to refs/heads/master by this push:
new 35f6f06a4 feat(ci): harden Rust crate publish chain after rc1 incident
(#3111)
35f6f06a4 is described below
commit 35f6f06a4d05d607c553b5e2f6514b4253d32a98
Author: Hubert Gruszecki <[email protected]>
AuthorDate: Tue Apr 14 09:35:44 2026 +0200
feat(ci): harden Rust crate publish chain after rc1 incident (#3111)
Apache Iggy 0.8.0 rc1 had to be cancelled because its crates.io
publish failed in five compounding ways: a path-only workspace dep
without a `version =` field broke `cargo publish`; the publish
order in CI was inverted (iggy_common ran before its dependency
iggy_binary_protocol); the wait-for-index loops silently fell
through on timeout, racing later steps against an unresolved dep;
git tags were created inline with the publish job, leaving zombie
tags whenever a mid-chain step failed; and PR CI had no mechanism
to catch any of this before merge. Each one was load-bearing on
its own; together they made the failure invisible until release
day. This change closes every one of those gaps in a single PR
so they cannot reappear independently.
Mark every workspace member that is not published to crates.io
as `publish = false`. Only four crates (iggy_binary_protocol,
iggy_common, iggy, iggy-cli) are uploaded by CI; the remaining
36 are docker images, internal libraries, tests, examples and
benchmarking tools. Setting `publish = false` documents the
intent and protects against `cargo publish --workspace` and
similar bulk tooling.
Add `scripts/verify-crates-publish.sh`, a self-contained shell
script invokable by both developers and CI. It spins up
cargo-http-registry on 127.0.0.1, patches the four workspace
deps to target the local registry, and runs `cargo publish`
on each crate in topological order. A trap reverts the patch,
removes the scratch `.cargo/config.toml` and stops the registry
on exit. The script catches the exact rc1 bug class: a missing
`version =` on a path dep fails `cargo publish` with
`dependency X does not specify a version`, exactly as crates.io
would reject it. Validated locally on both the success path and
a synthetic regression.
---
.github/actions/rust/pre-merge/action.yml | 14 +-
.github/config/components.yml | 20 ++
.github/workflows/_publish_rust_crates.yml | 94 +++++--
.github/workflows/publish.yml | 7 +-
bdd/rust/Cargo.toml | 1 +
core/ai/mcp/Cargo.toml | 1 +
core/bench/Cargo.toml | 1 +
core/bench/dashboard/frontend/Cargo.toml | 1 +
core/bench/dashboard/server/Cargo.toml | 1 +
core/bench/dashboard/shared/Cargo.toml | 1 +
core/bench/report/Cargo.toml | 1 +
core/bench/runner/Cargo.toml | 1 +
core/clock/Cargo.toml | 1 +
core/connectors/runtime/Cargo.toml | 1 +
core/connectors/sdk/Cargo.toml | 1 +
.../connectors/sinks/elasticsearch_sink/Cargo.toml | 1 +
core/connectors/sinks/http_sink/Cargo.toml | 1 +
core/connectors/sinks/iceberg_sink/Cargo.toml | 1 +
core/connectors/sinks/influxdb_sink/Cargo.toml | 1 +
core/connectors/sinks/mongodb_sink/Cargo.toml | 1 +
core/connectors/sinks/postgres_sink/Cargo.toml | 1 +
core/connectors/sinks/quickwit_sink/Cargo.toml | 1 +
core/connectors/sinks/stdout_sink/Cargo.toml | 1 +
.../sources/elasticsearch_source/Cargo.toml | 1 +
core/connectors/sources/influxdb_source/Cargo.toml | 1 +
core/connectors/sources/postgres_source/Cargo.toml | 1 +
core/connectors/sources/random_source/Cargo.toml | 1 +
core/consensus/Cargo.toml | 1 +
core/harness_derive/Cargo.toml | 1 +
core/integration/Cargo.toml | 1 +
core/journal/Cargo.toml | 1 +
core/message_bus/Cargo.toml | 1 +
core/metadata/Cargo.toml | 1 +
core/partitions/Cargo.toml | 1 +
core/server-ng/Cargo.toml | 1 +
core/server/Cargo.toml | 1 +
core/shard/Cargo.toml | 1 +
core/simulator/Cargo.toml | 1 +
core/tools/Cargo.toml | 1 +
examples/rust/Cargo.toml | 1 +
scripts/verify-crates-publish.sh | 295 +++++++++++++++++++++
41 files changed, 436 insertions(+), 30 deletions(-)
diff --git a/.github/actions/rust/pre-merge/action.yml
b/.github/actions/rust/pre-merge/action.yml
index 07f7cb152..11a429dc5 100644
--- a/.github/actions/rust/pre-merge/action.yml
+++ b/.github/actions/rust/pre-merge/action.yml
@@ -20,7 +20,7 @@ description: Rust pre-merge testing and linting github iggy
actions
inputs:
task:
- description: "Task to run (check, fmt, clippy, sort, machete, doctest,
test-1, test-2, compat)"
+ description: "Task to run (check, fmt, clippy, sort, machete, doctest,
verify-publish, test-1, test-2, compat)"
required: true
component:
description: "Component name (for context)"
@@ -56,6 +56,11 @@ runs:
machete)
cargo install cargo-machete --locked
;;
+ verify-publish)
+ if ! command -v cargo-http-registry >/dev/null 2>&1; then
+ cargo install cargo-http-registry --locked
+ fi
+ ;;
esac
shell: bash
@@ -150,6 +155,13 @@ runs:
cargo doc --no-deps --all-features --quiet
shell: bash
+ - name: Verify crates publish chain to local alt-registry
+ if: inputs.task == 'verify-publish'
+ env:
+ CA_BUNDLE: /etc/ssl/certs/ca-certificates.crt
+ run: ./scripts/verify-crates-publish.sh
+ shell: bash
+
- name: Install dependencies for Rust tests
if: startsWith(inputs.task, 'test-') && runner.os == 'Linux'
run: |
diff --git a/.github/config/components.yml b/.github/config/components.yml
index bb7a87f1b..e7b6dbbbc 100644
--- a/.github/config/components.yml
+++ b/.github/config/components.yml
@@ -24,6 +24,26 @@ components:
- "rust-toolchain.toml"
- ".cargo/**"
+ # crates.io publish chain verification. Runs
scripts/verify-crates-publish.sh,
+ # which spins up cargo-http-registry as a local alt-registry and
chain-publishes
+ # the four crates we ship to crates.io. Catches the class of bug that broke
+ # Apache Iggy 0.8.0 rc1 (path-only workspace dep without `version =`).
+ rust-publish:
+ depends_on:
+ - "rust-workspace" # rust-toolchain.toml / .cargo/** affect cargo publish
+ - "ci-infrastructure" # action.yml/workflow edits should re-run the check
+ paths:
+ - "Cargo.toml"
+ - "Cargo.lock"
+ - "core/binary_protocol/Cargo.toml"
+ - "core/common/Cargo.toml"
+ - "core/sdk/Cargo.toml"
+ - "core/cli/Cargo.toml"
+ - "scripts/verify-crates-publish.sh"
+ - ".github/actions/rust/pre-merge/action.yml"
+ tasks:
+ - "verify-publish"
+
# CI/CD infrastructure changes that require full regression
ci-infrastructure:
paths:
diff --git a/.github/workflows/_publish_rust_crates.yml
b/.github/workflows/_publish_rust_crates.yml
index b8cc787db..8c797f28d 100644
--- a/.github/workflows/_publish_rust_crates.yml
+++ b/.github/workflows/_publish_rust_crates.yml
@@ -112,49 +112,61 @@ jobs:
echo " iggy: $(scripts/extract-version.sh rust-sdk)"
echo " iggy-cli: $(scripts/extract-version.sh rust-cli)"
- # Step 1: Publish iggy_common
- - name: Publish iggy_common
- if: contains(inputs.crates, 'rust-common')
+ # Step 1: Publish iggy_binary_protocol (depends on nothing in-tree)
+ - name: Publish iggy_binary_protocol
+ if: contains(inputs.crates, 'rust-binary-protocol')
uses: ./.github/actions/rust/post-merge
with:
- package: iggy_common
- version: ${{ steps.versions.outputs.common }}
+ package: iggy_binary_protocol
+ version: ${{ steps.versions.outputs.protocol }}
dry_run: ${{ inputs.dry_run }}
- - name: Wait for iggy_common to be available
- if: contains(inputs.crates, 'rust-common') && inputs.dry_run == false
+ - name: Wait for iggy_binary_protocol to be available
+ if: contains(inputs.crates, 'rust-binary-protocol') && inputs.dry_run
== false
run: |
- echo "⏳ Waiting for iggy_common to be available on crates.io..."
+ echo "⏳ Waiting for iggy_binary_protocol to be available on
crates.io..."
+ found=false
for i in {1..30}; do
- if cargo search iggy_common --limit 1 | grep -q "^iggy_common =
\"${{ steps.versions.outputs.common }}\""; then
- echo "✅ iggy_common is now available"
+ if cargo search iggy_binary_protocol --limit 1 | grep -q
"^iggy_binary_protocol = \"${{ steps.versions.outputs.protocol }}\""; then
+ echo "✅ iggy_binary_protocol is now available"
+ found=true
break
fi
echo "Waiting... (attempt $i/30)"
sleep 10
done
+ if [ "$found" != "true" ]; then
+ echo "❌ Timed out waiting for iggy_binary_protocol on crates.io
index"
+ exit 1
+ fi
- # Step 2: Publish iggy_binary_protocol (depends on common)
- - name: Publish iggy_binary_protocol
- if: contains(inputs.crates, 'rust-binary-protocol')
+ # Step 2: Publish iggy_common (depends on iggy_binary_protocol)
+ - name: Publish iggy_common
+ if: contains(inputs.crates, 'rust-common')
uses: ./.github/actions/rust/post-merge
with:
- package: iggy_binary_protocol
- version: ${{ steps.versions.outputs.protocol }}
+ package: iggy_common
+ version: ${{ steps.versions.outputs.common }}
dry_run: ${{ inputs.dry_run }}
- - name: Wait for iggy_binary_protocol to be available
- if: contains(inputs.crates, 'rust-binary-protocol') && inputs.dry_run
== false
+ - name: Wait for iggy_common to be available
+ if: contains(inputs.crates, 'rust-common') && inputs.dry_run == false
run: |
- echo "⏳ Waiting for iggy_binary_protocol to be available on
crates.io..."
+ echo "⏳ Waiting for iggy_common to be available on crates.io..."
+ found=false
for i in {1..30}; do
- if cargo search iggy_binary_protocol --limit 1 | grep -q
"^iggy_binary_protocol = \"${{ steps.versions.outputs.protocol }}\""; then
- echo "✅ iggy_binary_protocol is now available"
+ if cargo search iggy_common --limit 1 | grep -q "^iggy_common =
\"${{ steps.versions.outputs.common }}\""; then
+ echo "✅ iggy_common is now available"
+ found=true
break
fi
echo "Waiting... (attempt $i/30)"
sleep 10
done
+ if [ "$found" != "true" ]; then
+ echo "❌ Timed out waiting for iggy_common on crates.io index"
+ exit 1
+ fi
# Step 3: Publish iggy SDK (depends on common and protocol)
- name: Publish iggy SDK
@@ -169,14 +181,20 @@ jobs:
if: contains(inputs.crates, 'rust-sdk') && inputs.dry_run == false
run: |
echo "⏳ Waiting for iggy to be available on crates.io..."
+ found=false
for i in {1..30}; do
if cargo search iggy --limit 1 | grep -q "^iggy = \"${{
steps.versions.outputs.sdk }}\""; then
echo "✅ iggy SDK is now available"
+ found=true
break
fi
echo "Waiting... (attempt $i/30)"
sleep 10
done
+ if [ "$found" != "true" ]; then
+ echo "❌ Timed out waiting for iggy SDK on crates.io index"
+ exit 1
+ fi
# Step 4: Publish iggy-cli (depends on SDK and protocol)
- name: Publish iggy-cli
@@ -187,9 +205,34 @@ jobs:
version: ${{ steps.versions.outputs.cli }}
dry_run: ${{ inputs.dry_run }}
- # Create git tags
- - name: Create git tags
- if: inputs.create_tags && inputs.dry_run == false
+ - name: Set final status output
+ id: final-status
+ if: always()
+ run: echo "status=${{ job.status }}" >> "$GITHUB_OUTPUT"
+
+ # Tag creation runs in a separate job so it only fires when every crate
+ # in the `publish` job has been uploaded successfully. Inlining this at
+ # the end of `publish` created zombie tags whenever a mid-chain publish
+ # step failed (leaving a version burned on crates.io with no matching
+ # git tag, which then prevented re-running the publish at that version).
+ create-tags:
+ name: Create git tags
+ needs: [publish]
+ runs-on: ubuntu-latest
+ if: success() && inputs.create_tags && inputs.dry_run == false
+ permissions:
+ contents: write
+ steps:
+ - name: Checkout
+ uses: actions/checkout@v4
+ with:
+ ref: ${{ inputs.commit || github.sha }}
+ # We only need the tip commit to run the tagging script, but tags
+ # must be available for the existence probe (`git rev-parse`) so
+ # we don't try to re-create tags that were already pushed.
+ fetch-depth: 1
+ fetch-tags: true
+ - name: Create and push tags
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
@@ -204,8 +247,3 @@ jobs:
echo "⏭️ Tag $TAG already exists"
fi
done
-
- - name: Set final status output
- id: final-status
- if: always()
- run: echo "status=${{ job.status }}" >> "$GITHUB_OUTPUT"
diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml
index ce72c73ae..5a73d5cac 100644
--- a/.github/workflows/publish.yml
+++ b/.github/workflows/publish.yml
@@ -118,8 +118,13 @@ permissions:
packages: write
id-token: write
+# Static group so two concurrent release dispatches cannot race the
+# crates.io / Maven Central / npm / NuGet / DockerHub / PyPI uploads.
+# `run_id` in the group was effectively a no-op because it is unique
+# per dispatch. `cancel-in-progress: false` keeps any in-flight upload
+# running to completion rather than leaving partial state in registries.
concurrency:
- group: publish-${{ github.run_id }}
+ group: publish-release
cancel-in-progress: false
jobs:
diff --git a/bdd/rust/Cargo.toml b/bdd/rust/Cargo.toml
index ec16e484f..ebedb19d2 100644
--- a/bdd/rust/Cargo.toml
+++ b/bdd/rust/Cargo.toml
@@ -21,6 +21,7 @@ name = "bdd"
version = "0.0.1"
edition = "2024"
license = "Apache-2.0"
+publish = false
[features]
bdd = []
diff --git a/core/ai/mcp/Cargo.toml b/core/ai/mcp/Cargo.toml
index 73ea944e8..9fdad8eb8 100644
--- a/core/ai/mcp/Cargo.toml
+++ b/core/ai/mcp/Cargo.toml
@@ -25,6 +25,7 @@ repository = "https://github.com/apache/iggy";
homepage = "https://iggy.apache.org";
keywords = ["iggy", "messaging", "streaming", "mcp"]
readme = "README.md"
+publish = false
[dependencies]
axum = { workspace = true }
diff --git a/core/bench/Cargo.toml b/core/bench/Cargo.toml
index 27daaa36e..3929a29fd 100644
--- a/core/bench/Cargo.toml
+++ b/core/bench/Cargo.toml
@@ -25,6 +25,7 @@ homepage = "https://iggy.apache.org";
description = "Benchmarking CLI for Iggy message streaming platform"
keywords = ["iggy", "cli", "messaging", "streaming"]
readme = "../../README.md"
+publish = false
[[bin]]
name = "iggy-bench"
diff --git a/core/bench/dashboard/frontend/Cargo.toml
b/core/bench/dashboard/frontend/Cargo.toml
index 606bfd3e2..d41268bea 100644
--- a/core/bench/dashboard/frontend/Cargo.toml
+++ b/core/bench/dashboard/frontend/Cargo.toml
@@ -20,6 +20,7 @@ name = "bench-dashboard-frontend"
license = "Apache-2.0"
version = "0.6.0"
edition = "2024"
+publish = false
[package.metadata.cargo-machete]
ignored = ["getrandom"]
diff --git a/core/bench/dashboard/server/Cargo.toml
b/core/bench/dashboard/server/Cargo.toml
index 60f66fd1f..c2cc96311 100644
--- a/core/bench/dashboard/server/Cargo.toml
+++ b/core/bench/dashboard/server/Cargo.toml
@@ -20,6 +20,7 @@ name = "iggy-bench-dashboard-server"
license = "Apache-2.0"
version = "0.7.0"
edition = "2024"
+publish = false
[dependencies]
actix-cors = { workspace = true }
diff --git a/core/bench/dashboard/shared/Cargo.toml
b/core/bench/dashboard/shared/Cargo.toml
index 670531a88..7f6637db7 100644
--- a/core/bench/dashboard/shared/Cargo.toml
+++ b/core/bench/dashboard/shared/Cargo.toml
@@ -20,6 +20,7 @@ name = "bench-dashboard-shared"
license = "Apache-2.0"
version = "0.1.0"
edition = "2024"
+publish = false
[dependencies]
bench-report = { workspace = true }
diff --git a/core/bench/report/Cargo.toml b/core/bench/report/Cargo.toml
index b82cb9b2f..cee78e6fb 100644
--- a/core/bench/report/Cargo.toml
+++ b/core/bench/report/Cargo.toml
@@ -21,6 +21,7 @@ version = "0.3.0"
edition = "2024"
description = "Benchmark report and chart generation library for iggy-bench
binary and iggy-benchmarks-dashboard web app"
license = "Apache-2.0"
+publish = false
[dependencies]
charming = { workspace = true }
diff --git a/core/bench/runner/Cargo.toml b/core/bench/runner/Cargo.toml
index c2b724c2a..0c981113e 100644
--- a/core/bench/runner/Cargo.toml
+++ b/core/bench/runner/Cargo.toml
@@ -20,6 +20,7 @@ name = "bench-runner"
license = "Apache-2.0"
version = "0.1.0"
edition = "2024"
+publish = false
[[bin]]
name = "iggy-bench-runner"
diff --git a/core/clock/Cargo.toml b/core/clock/Cargo.toml
index 844c65071..df6ad0a8c 100644
--- a/core/clock/Cargo.toml
+++ b/core/clock/Cargo.toml
@@ -19,6 +19,7 @@
name = "clock"
version = "0.1.0"
edition = "2024"
+publish = false
[dependencies]
iggy_common = { workspace = true }
diff --git a/core/connectors/runtime/Cargo.toml
b/core/connectors/runtime/Cargo.toml
index a6ccd3e2d..b81cf8016 100644
--- a/core/connectors/runtime/Cargo.toml
+++ b/core/connectors/runtime/Cargo.toml
@@ -27,6 +27,7 @@ homepage = "https://iggy.apache.org";
documentation = "https://iggy.apache.org/docs";
repository = "https://github.com/apache/iggy";
readme = "README.md"
+publish = false
[dependencies]
async-trait = { workspace = true }
diff --git a/core/connectors/sdk/Cargo.toml b/core/connectors/sdk/Cargo.toml
index 846a68ff4..364b7e6be 100644
--- a/core/connectors/sdk/Cargo.toml
+++ b/core/connectors/sdk/Cargo.toml
@@ -27,6 +27,7 @@ homepage = "https://iggy.apache.org";
documentation = "https://iggy.apache.org/docs";
repository = "https://github.com/apache/iggy";
readme = "../../README.md"
+publish = false
[package.metadata.cargo-machete]
ignored = ["prost"]
diff --git a/core/connectors/sinks/elasticsearch_sink/Cargo.toml
b/core/connectors/sinks/elasticsearch_sink/Cargo.toml
index 5557eb6c4..b01f61ddc 100644
--- a/core/connectors/sinks/elasticsearch_sink/Cargo.toml
+++ b/core/connectors/sinks/elasticsearch_sink/Cargo.toml
@@ -27,6 +27,7 @@ homepage = "https://iggy.apache.org";
documentation = "https://iggy.apache.org/docs";
repository = "https://github.com/apache/iggy";
readme = "../../README.md"
+publish = false
[package.metadata.cargo-machete]
ignored = ["dashmap", "once_cell", "futures", "simd-json"]
diff --git a/core/connectors/sinks/http_sink/Cargo.toml
b/core/connectors/sinks/http_sink/Cargo.toml
index f27921adc..dc2d2890c 100644
--- a/core/connectors/sinks/http_sink/Cargo.toml
+++ b/core/connectors/sinks/http_sink/Cargo.toml
@@ -27,6 +27,7 @@ homepage = "https://iggy.apache.org";
documentation = "https://iggy.apache.org/docs";
repository = "https://github.com/apache/iggy";
readme = "../../README.md"
+publish = false
[lib]
crate-type = ["cdylib", "lib"]
diff --git a/core/connectors/sinks/iceberg_sink/Cargo.toml
b/core/connectors/sinks/iceberg_sink/Cargo.toml
index 4b5c5a974..5b5d1d89c 100644
--- a/core/connectors/sinks/iceberg_sink/Cargo.toml
+++ b/core/connectors/sinks/iceberg_sink/Cargo.toml
@@ -26,6 +26,7 @@ homepage = "https://iggy.apache.org";
documentation = "https://iggy.apache.org/docs";
repository = "https://github.com/apache/iggy";
readme = "../../README.md"
+publish = false
[package.metadata.cargo-machete]
ignored = ["dashmap", "once_cell"]
diff --git a/core/connectors/sinks/influxdb_sink/Cargo.toml
b/core/connectors/sinks/influxdb_sink/Cargo.toml
index 7b51f8256..30360558c 100644
--- a/core/connectors/sinks/influxdb_sink/Cargo.toml
+++ b/core/connectors/sinks/influxdb_sink/Cargo.toml
@@ -27,6 +27,7 @@ homepage = "https://iggy.apache.org";
documentation = "https://iggy.apache.org/docs";
repository = "https://github.com/apache/iggy";
readme = "../../README.md"
+publish = false
[package.metadata.cargo-machete]
ignored = ["dashmap", "once_cell", "futures"]
diff --git a/core/connectors/sinks/mongodb_sink/Cargo.toml
b/core/connectors/sinks/mongodb_sink/Cargo.toml
index b2e0856e0..00e0f3d40 100644
--- a/core/connectors/sinks/mongodb_sink/Cargo.toml
+++ b/core/connectors/sinks/mongodb_sink/Cargo.toml
@@ -27,6 +27,7 @@ homepage = "https://iggy.apache.org";
documentation = "https://iggy.apache.org/docs";
repository = "https://github.com/apache/iggy";
readme = "../../README.md"
+publish = false
[lib]
crate-type = ["cdylib", "lib"]
diff --git a/core/connectors/sinks/postgres_sink/Cargo.toml
b/core/connectors/sinks/postgres_sink/Cargo.toml
index f640fa3e2..70d7c78dd 100644
--- a/core/connectors/sinks/postgres_sink/Cargo.toml
+++ b/core/connectors/sinks/postgres_sink/Cargo.toml
@@ -27,6 +27,7 @@ homepage = "https://iggy.apache.org";
documentation = "https://iggy.apache.org/docs";
repository = "https://github.com/apache/iggy";
readme = "../../README.md"
+publish = false
[package.metadata.cargo-machete]
ignored = ["dashmap", "once_cell", "futures", "simd-json", "prost"]
diff --git a/core/connectors/sinks/quickwit_sink/Cargo.toml
b/core/connectors/sinks/quickwit_sink/Cargo.toml
index 776810daf..6bdd19fb6 100644
--- a/core/connectors/sinks/quickwit_sink/Cargo.toml
+++ b/core/connectors/sinks/quickwit_sink/Cargo.toml
@@ -27,6 +27,7 @@ homepage = "https://iggy.apache.org";
documentation = "https://iggy.apache.org/docs";
repository = "https://github.com/apache/iggy";
readme = "../../README.md"
+publish = false
[package.metadata.cargo-machete]
ignored = ["dashmap", "once_cell"]
diff --git a/core/connectors/sinks/stdout_sink/Cargo.toml
b/core/connectors/sinks/stdout_sink/Cargo.toml
index 354d287ce..c6d81878d 100644
--- a/core/connectors/sinks/stdout_sink/Cargo.toml
+++ b/core/connectors/sinks/stdout_sink/Cargo.toml
@@ -27,6 +27,7 @@ homepage = "https://iggy.apache.org";
documentation = "https://iggy.apache.org/docs";
repository = "https://github.com/apache/iggy";
readme = "../../README.md"
+publish = false
[package.metadata.cargo-machete]
ignored = ["dashmap", "once_cell"]
diff --git a/core/connectors/sources/elasticsearch_source/Cargo.toml
b/core/connectors/sources/elasticsearch_source/Cargo.toml
index 6a4604522..f23f60b47 100644
--- a/core/connectors/sources/elasticsearch_source/Cargo.toml
+++ b/core/connectors/sources/elasticsearch_source/Cargo.toml
@@ -27,6 +27,7 @@ homepage = "https://iggy.apache.org";
documentation = "https://iggy.apache.org/docs";
repository = "https://github.com/apache/iggy";
readme = "../../README.md"
+publish = false
[package.metadata.cargo-machete]
ignored = ["dashmap", "once_cell", "futures", "simd-json"]
diff --git a/core/connectors/sources/influxdb_source/Cargo.toml
b/core/connectors/sources/influxdb_source/Cargo.toml
index c89a440b1..b532d5aaa 100644
--- a/core/connectors/sources/influxdb_source/Cargo.toml
+++ b/core/connectors/sources/influxdb_source/Cargo.toml
@@ -27,6 +27,7 @@ homepage = "https://iggy.apache.org";
documentation = "https://iggy.apache.org/docs";
repository = "https://github.com/apache/iggy";
readme = "../../README.md"
+publish = false
[package.metadata.cargo-machete]
ignored = ["dashmap", "once_cell", "futures"]
diff --git a/core/connectors/sources/postgres_source/Cargo.toml
b/core/connectors/sources/postgres_source/Cargo.toml
index a42e5ff29..0490b1a30 100644
--- a/core/connectors/sources/postgres_source/Cargo.toml
+++ b/core/connectors/sources/postgres_source/Cargo.toml
@@ -27,6 +27,7 @@ homepage = "https://iggy.apache.org";
documentation = "https://iggy.apache.org/docs";
repository = "https://github.com/apache/iggy";
readme = "../../README.md"
+publish = false
[package.metadata.cargo-machete]
ignored = ["dashmap", "once_cell", "futures", "simd-json"]
diff --git a/core/connectors/sources/random_source/Cargo.toml
b/core/connectors/sources/random_source/Cargo.toml
index 7123c6770..b2e8341d1 100644
--- a/core/connectors/sources/random_source/Cargo.toml
+++ b/core/connectors/sources/random_source/Cargo.toml
@@ -27,6 +27,7 @@ homepage = "https://iggy.apache.org";
documentation = "https://iggy.apache.org/docs";
repository = "https://github.com/apache/iggy";
readme = "../../README.md"
+publish = false
[package.metadata.cargo-machete]
ignored = ["dashmap", "once_cell"]
diff --git a/core/consensus/Cargo.toml b/core/consensus/Cargo.toml
index 89a4baff1..40d503aa1 100644
--- a/core/consensus/Cargo.toml
+++ b/core/consensus/Cargo.toml
@@ -26,6 +26,7 @@ homepage = "https://iggy.apache.org";
documentation = "https://iggy.apache.org/docs";
repository = "https://github.com/apache/iggy";
readme = "../../../README.md"
+publish = false
[dependencies]
bit-set = { workspace = true }
diff --git a/core/harness_derive/Cargo.toml b/core/harness_derive/Cargo.toml
index c9ed19f02..4dcd3f77b 100644
--- a/core/harness_derive/Cargo.toml
+++ b/core/harness_derive/Cargo.toml
@@ -20,6 +20,7 @@ name = "harness_derive"
version = "0.1.0"
edition = "2024"
license = "Apache-2.0"
+publish = false
[lib]
proc-macro = true
diff --git a/core/integration/Cargo.toml b/core/integration/Cargo.toml
index 9b5a993df..950d39093 100644
--- a/core/integration/Cargo.toml
+++ b/core/integration/Cargo.toml
@@ -20,6 +20,7 @@ name = "integration"
version = "0.0.1"
edition = "2024"
license = "Apache-2.0"
+publish = false
# Some tests are failing in CI due to lack of IPv6 interfaces
# inside the docker containers. This is a temporary workaround (hopefully).
diff --git a/core/journal/Cargo.toml b/core/journal/Cargo.toml
index 1f255cc5c..66c4853c1 100644
--- a/core/journal/Cargo.toml
+++ b/core/journal/Cargo.toml
@@ -26,6 +26,7 @@ homepage = "https://iggy.apache.org";
documentation = "https://iggy.apache.org/docs";
repository = "https://github.com/apache/iggy";
readme = "../../../README.md"
+publish = false
[dependencies]
bytemuck = { workspace = true }
diff --git a/core/message_bus/Cargo.toml b/core/message_bus/Cargo.toml
index d43225bfb..e31049902 100644
--- a/core/message_bus/Cargo.toml
+++ b/core/message_bus/Cargo.toml
@@ -26,6 +26,7 @@ homepage = "https://iggy.apache.org";
documentation = "https://iggy.apache.org/docs";
repository = "https://github.com/apache/iggy";
readme = "../../../README.md"
+publish = false
[dependencies]
iggy_binary_protocol = { workspace = true }
diff --git a/core/metadata/Cargo.toml b/core/metadata/Cargo.toml
index d579b70fe..c437c7efa 100644
--- a/core/metadata/Cargo.toml
+++ b/core/metadata/Cargo.toml
@@ -26,6 +26,7 @@ homepage = "https://iggy.apache.org";
documentation = "https://iggy.apache.org/docs";
repository = "https://github.com/apache/iggy";
readme = "../../../README.md"
+publish = false
[dependencies]
ahash = { workspace = true }
diff --git a/core/partitions/Cargo.toml b/core/partitions/Cargo.toml
index 7d1db6c7f..158790862 100644
--- a/core/partitions/Cargo.toml
+++ b/core/partitions/Cargo.toml
@@ -26,6 +26,7 @@ homepage = "https://iggy.apache.org";
documentation = "https://iggy.apache.org/docs";
repository = "https://github.com/apache/iggy";
readme = "../../README.md"
+publish = false
[dependencies]
bytemuck = { workspace = true }
diff --git a/core/server-ng/Cargo.toml b/core/server-ng/Cargo.toml
index 7875d2286..79211fe7c 100644
--- a/core/server-ng/Cargo.toml
+++ b/core/server-ng/Cargo.toml
@@ -20,6 +20,7 @@ name = "server-ng"
version = "0.8.0"
edition = "2024"
license = "Apache-2.0"
+publish = false
[package.metadata.cargo-udeps.ignore]
normal = ["tracing-appender"]
diff --git a/core/server/Cargo.toml b/core/server/Cargo.toml
index 3125c43b3..656b5480f 100644
--- a/core/server/Cargo.toml
+++ b/core/server/Cargo.toml
@@ -20,6 +20,7 @@ name = "server"
version = "0.8.0"
edition = "2024"
license = "Apache-2.0"
+publish = false
[package.metadata.cargo-udeps.ignore]
normal = ["tracing-appender"]
diff --git a/core/shard/Cargo.toml b/core/shard/Cargo.toml
index f5cb07c4e..767c541c3 100644
--- a/core/shard/Cargo.toml
+++ b/core/shard/Cargo.toml
@@ -19,6 +19,7 @@
name = "shard"
version = "0.1.0"
edition = "2024"
+publish = false
[dependencies]
bytes = { workspace = true }
diff --git a/core/simulator/Cargo.toml b/core/simulator/Cargo.toml
index 556e47388..9f9d264e0 100644
--- a/core/simulator/Cargo.toml
+++ b/core/simulator/Cargo.toml
@@ -19,6 +19,7 @@
name = "simulator"
version = "0.1.0"
edition = "2024"
+publish = false
[dependencies]
bytemuck = { workspace = true }
diff --git a/core/tools/Cargo.toml b/core/tools/Cargo.toml
index 55cc857e2..e262a02b6 100644
--- a/core/tools/Cargo.toml
+++ b/core/tools/Cargo.toml
@@ -20,6 +20,7 @@ name = "tools"
version = "0.1.0"
edition = "2024"
license = "Apache-2.0"
+publish = false
[[bin]]
name = "data-seeder-tool"
diff --git a/examples/rust/Cargo.toml b/examples/rust/Cargo.toml
index 67005fd50..2027745aa 100644
--- a/examples/rust/Cargo.toml
+++ b/examples/rust/Cargo.toml
@@ -20,6 +20,7 @@ name = "iggy_examples"
version = "0.0.6"
edition = "2024"
license = "Apache-2.0"
+publish = false
[dependencies]
anyhow = { workspace = true }
diff --git a/scripts/verify-crates-publish.sh b/scripts/verify-crates-publish.sh
new file mode 100755
index 000000000..6464cae86
--- /dev/null
+++ b/scripts/verify-crates-publish.sh
@@ -0,0 +1,295 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# End-to-end verification that the four published Rust crates can actually
+# be uploaded to a crates.io-like registry. Spins up cargo-http-registry as
+# a local alt-registry, then publishes iggy_binary_protocol, iggy_common,
+# iggy and iggy-cli in topological order. Catches missing `version = "..."`
+# on path deps, wrong publish order, and any manifest-level regression that
+# only surfaces at `cargo publish` time (the exact class of bug that broke
+# Apache Iggy 0.8.0 rc1).
+#
+# This script is invoked both by developers (locally) and by CI. It must
+# leave the working tree untouched on exit, even on failure.
+#
+# Usage:
+# scripts/verify-crates-publish.sh
+#
+# Dependencies (install manually, or re-run with AUTO_INSTALL=1):
+# cargo-http-registry (cargo install cargo-http-registry --locked)
+#
+# Environment variables (all optional):
+# REGISTRY_PORT Port for the local registry (default: 35503)
+# CA_BUNDLE Path to the system CA bundle used by libgit2.
+# Auto-detected on common Linux distros if unset.
+# AUTO_INSTALL If "1", install cargo-http-registry (default: 0)
+#
+# Exit codes:
+# 0 success, all four crates published to the local registry
+# 1 tooling missing or preflight failure
+# 2 publish chain failed on one of the four crates
+
+set -euo pipefail
+
+REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+cd "$REPO_ROOT"
+
+REGISTRY_PORT="${REGISTRY_PORT:-35503}"
+REGISTRY_DIR="$(mktemp -d -t iggy-local-reg.XXXXXX)"
+REGISTRY_LOG="$(mktemp -t iggy-local-reg-log.XXXXXX)"
+# Byte-exact snapshot of Cargo.toml. Restored by cleanup() instead of
+# `git checkout`, so the revert is independent of git state and cannot
+# eat an in-flight edit if the preflight dirty check is ever bypassed.
+CARGO_TOML_BACKUP="$(mktemp -t iggy-cargo-toml.XXXXXX)"
+cp Cargo.toml "$CARGO_TOML_BACKUP"
+AUTO_INSTALL="${AUTO_INSTALL:-0}"
+
+# Ordered topologically: each crate depends only on the ones before it.
+CRATES=(
+ iggy_binary_protocol
+ iggy_common
+ iggy
+ iggy-cli
+)
+
+REGISTRY_PID=""
+CARGO_CONFIG_CREATED=false
+CARGO_TOML_PATCHED=false
+
+log() { printf '>>> %s\n' "$*"; }
+err() { printf '!!! %s\n' "$*" >&2; }
+
+# Invoked via `trap ... EXIT INT TERM`; shellcheck can't see indirect calls.
+# shellcheck disable=SC2329
+cleanup() {
+ local ec=$?
+ set +e
+ trap - EXIT INT TERM
+
+ if [[ "$CARGO_TOML_PATCHED" == "true" ]]; then
+ log "Reverting Cargo.toml patch"
+ cp "$CARGO_TOML_BACKUP" Cargo.toml
+ fi
+ rm -f "$CARGO_TOML_BACKUP"
+
+ if [[ "$CARGO_CONFIG_CREATED" == "true" ]]; then
+ log "Removing scratch .cargo/config.toml"
+ rm -f .cargo/config.toml
+ rmdir .cargo 2>/dev/null || true
+ fi
+
+ if [[ -n "$REGISTRY_PID" ]] && kill -0 "$REGISTRY_PID" 2>/dev/null; then
+ log "Stopping local registry (pid $REGISTRY_PID)"
+ kill -TERM "$REGISTRY_PID" 2>/dev/null || true
+ # Give it half a second then force
+ for _ in 1 2 3 4 5; do
+ kill -0 "$REGISTRY_PID" 2>/dev/null || break
+ sleep 0.1
+ done
+ kill -KILL "$REGISTRY_PID" 2>/dev/null || true
+ wait "$REGISTRY_PID" 2>/dev/null || true
+ fi
+
+ rm -rf "$REGISTRY_DIR" "$REGISTRY_LOG"
+ exit "$ec"
+}
+trap cleanup EXIT INT TERM
+
+# ---------------------------------------------------------------------------
+# 0. Preconditions
+# ---------------------------------------------------------------------------
+if ! command -v cargo >/dev/null 2>&1; then
+ err "cargo is not on PATH"
+ exit 1
+fi
+
+if ! command -v cargo-http-registry >/dev/null 2>&1; then
+ if [[ "$AUTO_INSTALL" == "1" ]]; then
+ log "Installing cargo-http-registry"
+ cargo install cargo-http-registry --locked
+ else
+ err "cargo-http-registry is not installed"
+ err "Install it with: cargo install cargo-http-registry --locked"
+ err "Or re-run this script with AUTO_INSTALL=1"
+ exit 1
+ fi
+fi
+
+# libgit2 inside cargo needs an explicit CA bundle on many Linux distros.
+# Auto-detect common locations when the caller hasn't overridden it.
+if [[ -z "${CA_BUNDLE:-}" ]]; then
+ for candidate in \
+ /etc/ssl/certs/ca-certificates.crt \
+ /etc/pki/tls/certs/ca-bundle.crt \
+ /etc/ssl/cert.pem \
+ /etc/ssl/ca-bundle.pem \
+ /opt/homebrew/etc/ca-certificates/cert.pem \
+ /usr/local/etc/ca-certificates/cert.pem
+ do
+ if [[ -r "$candidate" ]]; then
+ CA_BUNDLE="$candidate"
+ break
+ fi
+ done
+fi
+
+if [[ -z "${CA_BUNDLE:-}" || ! -r "$CA_BUNDLE" ]]; then
+ err "Could not locate a system CA bundle. Set CA_BUNDLE=/path/to/ca.crt"
+ exit 1
+fi
+log "Using CA bundle: $CA_BUNDLE"
+
+if [[ -e .cargo/config.toml ]]; then
+ err ".cargo/config.toml already exists; refusing to overwrite"
+ err "Move it aside before running this script"
+ exit 1
+fi
+
+# Refuse to run on top of in-flight Cargo.toml edits. The script patches
+# this file in step 3, and we don't want to mix caller edits with the
+# scratch registry patch: a successful run would happily roll both
+# forward, and a failed run would force the caller to disentangle them.
+if ! git diff --quiet HEAD -- Cargo.toml; then
+ err "Cargo.toml has uncommitted changes; refusing to patch"
+ err "Commit or stash your changes before running this script"
+ exit 1
+fi
+
+# ---------------------------------------------------------------------------
+# 1. Start the local registry
+# ---------------------------------------------------------------------------
+log "Starting cargo-http-registry at 127.0.0.1:${REGISTRY_PORT}"
+cargo-http-registry --addr "127.0.0.1:${REGISTRY_PORT}" "$REGISTRY_DIR" \
+ > "$REGISTRY_LOG" 2>&1 &
+REGISTRY_PID=$!
+
+# cargo-http-registry speaks git-http; `git ls-remote` is the closest
+# thing to a health check. A GET on any URL path returns 405 because the
+# git-upload-pack endpoint expects POST, so don't probe with plain curl.
+for attempt in $(seq 1 20); do
+ if git ls-remote "http://127.0.0.1:${REGISTRY_PORT}/git"; HEAD >/dev/null
2>&1; then
+ log "Registry is up (pid $REGISTRY_PID)"
+ break
+ fi
+ if ! kill -0 "$REGISTRY_PID" 2>/dev/null; then
+ err "cargo-http-registry died before becoming ready"
+ cat "$REGISTRY_LOG" >&2 || true
+ exit 1
+ fi
+ sleep 0.5
+ if (( attempt == 20 )); then
+ err "Registry failed to become ready within 10s"
+ cat "$REGISTRY_LOG" >&2 || true
+ exit 1
+ fi
+done
+
+# ---------------------------------------------------------------------------
+# 2. Write a local .cargo/config.toml pointing at the registry
+# ---------------------------------------------------------------------------
+log "Writing .cargo/config.toml"
+mkdir -p .cargo
+CARGO_CONFIG_CREATED=true
+cat > .cargo/config.toml <<EOF
+[registries.local-dev]
+index = "http://127.0.0.1:${REGISTRY_PORT}/git";
+
+[net]
+git-fetch-with-cli = true
+
+[http]
+cainfo = "${CA_BUNDLE}"
+EOF
+
+# ---------------------------------------------------------------------------
+# 3. Patch root Cargo.toml so path deps on the four iggy crates resolve
+# against local-dev during `cargo publish`. The edit is reverted by the
+# cleanup trap, whether the script succeeds or fails.
+# ---------------------------------------------------------------------------
+log "Patching workspace.dependencies to target local-dev"
+CARGO_TOML_PATCHED=true
+# Injects `, registry = "local-dev"` before the closing brace of the line.
+# Match by crate name + path + version so we do not touch unrelated entries.
+sed -i -E \
+ -e 's|^(iggy_binary_protocol = \{ path = "[^"]+", version = "[^"]+")(
\})$|\1, registry = "local-dev"\2|' \
+ -e 's|^(iggy_common = \{ path = "[^"]+", version = "[^"]+")( \})$|\1,
registry = "local-dev"\2|' \
+ -e 's|^(iggy = \{ path = "[^"]+", version = "[^"]+")( \})$|\1, registry =
"local-dev"\2|' \
+ -e 's|^(iggy-cli = \{ path = "[^"]+", version = "[^"]+")( \})$|\1,
registry = "local-dev"\2|' \
+ Cargo.toml
+
+# Sanity check: all four lines must now carry the registry marker. This
+# catches formatting drift in Cargo.toml before it becomes a confusing
+# publish-time error.
+missing=0
+for crate in "${CRATES[@]}"; do
+ if ! grep -Eq "^${crate} = \{.*registry = \"local-dev\".*\}$" Cargo.toml;
then
+ err "Failed to patch workspace dep for ${crate}"
+ missing=1
+ fi
+done
+if (( missing )); then
+ err "One or more workspace deps did not match the expected format"
+ err "Expected: <crate> = { path = \"...\", version = \"...\" }"
+ err "Current state:"
+ grep -nE "^(iggy|iggy-cli|iggy_binary_protocol|iggy_common) = " Cargo.toml
>&2 || true
+ exit 1
+fi
+
+# ---------------------------------------------------------------------------
+# 4. Publish the four crates to local-dev in topological order
+# ---------------------------------------------------------------------------
+# --no-verify: the PR build pipeline already compiles the workspace; this
+# job only checks the publish path. Skipping verify means we
+# don't need the transitive crates.io dep graph to be locally
+# resolvable.
+# --allow-dirty: we just patched Cargo.toml; the working tree is dirty by
+# design. The cleanup trap reverts the patch on exit.
+for crate in "${CRATES[@]}"; do
+ log "Publishing ${crate} to local-dev"
+ if ! cargo publish \
+ -p "$crate" \
+ --registry local-dev \
+ --token ci \
+ --allow-dirty \
+ --no-verify
+ then
+ err "Publish failed for ${crate}"
+ exit 2
+ fi
+done
+
+# ---------------------------------------------------------------------------
+# 5. Cross-check: every crate's tarball must actually exist on the registry
+# ---------------------------------------------------------------------------
+log "Verifying all four crates appear in the local registry"
+# crates-index path layout for 4+ char names: "<first-2>/<next-2>/<name>".
+# Every published iggy crate is >=4 chars, so the shorter-name buckets
+# (1/, 2/, 3/<c>/) from the full layout are intentionally not handled.
+for crate in "${CRATES[@]}"; do
+ lc="${crate,,}"
+ rel="${lc:0:2}/${lc:2:2}/$lc"
+ if [[ ! -f "$REGISTRY_DIR/$rel" ]]; then
+ err "Crate ${crate} missing from local registry (expected at $rel)"
+ ls -la "$REGISTRY_DIR" >&2 || true
+ exit 2
+ fi
+done
+
+log "All four crates published successfully to local-dev"
+log "Registry directory: $REGISTRY_DIR (will be cleaned up)"
+exit 0
