+1 On Thu, Sep 3, 2026 at 4:10 AM Yang Jie <[email protected]> wrote:
> +1 > > On 2026/09/03 01:12:00 bo yang wrote: > > +1 (non binding) > > > > > > On Wed, Sep 2, 2026 at 5:36 PM Ruifeng Zheng <[email protected]> wrote: > > > > > +1 > > > > > > On Thu, Sep 3, 2026 at 6:07 AM L. C. Hsieh <[email protected]> wrote: > > > > > >> +1, on the technical merits. > > >> > > >> I verified RC2 end-to-end and the software is in good shape — build, > > >> tests, and > > >> functional results are all correct (details below). Two > > >> packaging/release-policy > > >> items stood out that I'd ask the RM to look at before this is > finalized; > > >> I don't > > >> think either is a code problem, but the first in particular may matter > > >> for a > > >> formal ASF release, so I'm noting them rather than blocking. > > >> > > >> ## What I verified (all passed) > > >> > > >> Identity & integrity: > > >> - git tag v4.2.0-rc2 resolves to commit ce1fa08 (matches the vote > email). > > >> - Downloaded macOS universal2 wheel and the sdist; both SHA-256 match > the > > >> digests GitHub recorded for the assets. > > >> - proto pinning in the tree is PROTO_VERSION=v4.2.0 / > SPARK_SHA=32f7299, > > >> consistent with targeting Spark 4.2.0. > > >> > > >> Python wheel (installed into a throwaway venv): > > >> - The macOS universal2 wheel installs and imports as a drop-in > `pyspark` > > >> (pyspark-client-rust 4.2.0). The abi3 native extension is a genuine > > >> universal2 binary (x86_64 + arm64 slices) and loads on Apple > Silicon. > > >> - Ran real workloads against a local Spark 4.2.0 Connect server (the > > >> bundled > > >> spark-connect_2.13-4.2.0.jar from the apache/spark:4.2.0 image): > > >> range/count, > > >> groupBy aggregation, SQL, createDataFrame + filter — all correct; > > >> server-reported spark.version = 4.2.0. > > >> - collect() and toPandas() type conversion is correct across > > >> int/string/double/ > > >> boolean and NULL->NaN. > > >> > > >> Rust crates (from the tag): > > >> - cargo build --release and cargo test --release both pass, no > failures > > >> (server-dependent e2e tests are ignored by design without a live > > >> server). > > >> - The Rust `sql` example runs end-to-end against the live server (SQL > -> > > >> write > > >> parquet -> read back), confirming the native tonic transport works > > >> against a > > >> real Spark 4.2.0 server. > > >> > > >> ## Notes for the RM > > >> > > >> 1. The candidate is not published to the ASF distribution area and is > not > > >> signed. The release files live only on GitHub Releases. There > > >> is no .asc PGP signature, no .sha512, and no reference to the Spark > > >> KEYS > > >> file. GitHub's SHA-256 digests establish transfer integrity only, > not > > >> that > > >> a release manager signed the artifacts. ASF policy is that the > thing > > >> being > > >> voted on is a signed source release in the dist area; convenience > > >> binaries > > >> are not the subject of the vote. Notably the sibling Spark Connect > Go > > >> client > > >> RC does publish to dist.apache.org, so there is an established > > >> pattern here > > >> for this candidate to follow. > > >> > > >> 2. The source distribution does not ship a LICENSE. The sdist > > >> (pyspark_client_rust-4.2.0.tar.gz) has 390 files and none of them > is a > > >> LICENSE file, so downstream repackagers get no license text — this > one > > >> should be fixed regardless. Separately, the tree bundles some > vendored > > >> third-party code, so a NOTICE also looks like it's needed here (and > > >> there > > >> isn't one); worth the RM double-checking the LICENSE/NOTICE against > > >> what's > > >> actually bundled. > > >> > > >> ## Minor (non-blocking) observation > > >> > > >> DataFrame.show() renders rows in a compact form (e.g. `[apple, 123]`) > > >> rather > > >> than PySpark's bordered ASCII table. Values are correct; flagging only > > >> because > > >> the client is presented as a strict drop-in and some users scrape > show() > > >> output. > > >> > > >> > > >> Thanks for driving this — the technical work looks solid. > > >> > > >> > > >> On Tue, Sep 1, 2026 at 3:08 PM Hyukjin Kwon <[email protected]> > wrote: > > >> > > >>> Please vote on releasing the following candidate as Apache Spark > Connect > > >>> Rust > > >>> Client 4.2.0 RC2. > > >>> > > >>> This is the *first* release of the ground-up Rust rewrite of the > Spark > > >>> Connect > > >>> Python client (published to PyPI as pyspark-client-rust): a drop-in > > >>> replacement > > >>> for pyspark-client <https://pypi.org/project/pyspark-client/>, > backed > > >>> by a native Rust core instead of py4j/grpcio. > > >>> > > >>> The vote is open until Thursday, September 7, 2026, 5:00 PM PDT and > > >>> passes if a majority of +1 PMC votes are cast, with a minimum of > > >>> 3 +1 votes. > > >>> > > >>> [ ] +1 Release this package as Apache Spark Connect Rust Client 4.2.0 > > >>> [ ] -1 Do not release this package because … > > >>> > > >>> TAG: > > >>> https://github.com/apache/spark-connect-rust/releases/tag/v4.2.0-rc2 > > >>> (commit: ce1fa08) > > >>> > > >>> RELEASE FILES (wheels + sdist, built and attached by GitHub Actions): > > >>> https://github.com/apache/spark-connect-rust/releases/tag/v4.2.0-rc2 > > >>> > > >>> DOCUMENTATION: > > >>> https://apache.github.io/spark-connect-rust/ > > >>> > > >>> LIST OF ISSUES: > > >>> https://issues.apache.org/jira/projects/SPARK/versions/12357484 > > >>> HOW TO TRY IT — PYTHON (pip) > > >>> > > >>> The release attaches abi3 wheels (Linux x86_64/aarch64, macOS > universal2, > > >>> Windows x86_64; CPython 3.9+) plus a source distribution. Grab the > wheel > > >>> for > > >>> your platform from the release page and install it directly, e.g.: > > >>> > > >>> pip install > https://github.com/apache/spark-connect-rust/releases/download/v4.2.0-rc2/pyspark_client_rust-4.2.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl > > >>> > > >>> Then use it exactly like PySpark. See if your existing workloads work > > >>> with this. > > >>> HOW TO TEST IT — RUST (cargo) > > >>> > > >>> The Rust crates are intentionally NOT on crates.io for a release > > >>> candidate > > >>> (crates.io is immutable; the crates are published there only when > the > > >>> release is > > >>> finalized). Test against the tagged source instead: > > >>> > > >>> # A) Pin a git dependency to the RC tag in your project's > Cargo.toml: > > >>> [dependencies] > > >>> apache-spark-connect = { git = " > https://github.com/apache/spark-connect-rust";, tag = "v4.2.0-rc2" } > > >>> > > >>> # B) Or build and run the crates' own tests from the tag: > > >>> git clone --branch v4.2.0-rc2 > https://github.com/apache/spark-connect-rust > > >>> cd spark-connect-rust > > >>> cargo build --release > > >>> cargo test > > >>> > > >>> HIGHLIGHTS > > >>> > > >>> - *Drop-in PySpark Connect Python API*, verified against real > > >>> PySpark 4.2.0 two > > >>> ways: the official PySpark test suite runs against this client, > and a > > >>> behavioral API-parity sweep compares the Python API surface > > >>> (functions and > > >>> DataFrame / Column / SparkSession / GroupedData / Window methods) > > >>> call-by-call > > >>> to PySpark. > > >>> - Spark Connect Rust API *full* coverage, including *Rust UDF*s > > >>> (docs: https://apache.github.io/spark-connect-rust/). > > >>> - *Native Rust transport* via tonic; cloudpickle vendored to match > > >>> upstream, so > > >>> no grpcio/py4j at runtime. and *native type conversion.* > > >>> > > >>> WHY DOES THE VERSION START AT 4.2.0? > > >>> > > >>> Even though this is the first release of the rewrite, the version > starts > > >>> at > > >>> 4.2.0 to track the Apache Spark / PySpark version it targets. This > (1) > > >>> removes > > >>> any ambiguity about which Spark version a given client is compatible > > >>> with, and > > >>> (2) lets PySpark users pip install pyspark-client-rust== exactly > > >>> as they would pyspark-client, keeping the same versioning across > both. > > >>> > > >>> Thanks, > > >>> Hyukjin Kwon > > >>> > > >> > > > > > > -- > > > Ruifeng Zheng > > > E-mail: [email protected] > > > > > > > --------------------------------------------------------------------- > To unsubscribe e-mail: [email protected] > >
